00010000:00010000:3.0:1789760748.790097:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.800378:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68722688 -> 68726784 00002000:00010000:3.0:1789760748.800384:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134248 -> 134256 00010000:00010000:3.0:1789760748.800453:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.800476:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.800487:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.800495:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760748.800501:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.800511:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.800545:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.800554:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.800690:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503532032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10713us (11046us total) trans 4294984085 rc 0/0 00000100:00100000:3.0:1789760748.800703:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16789 00000100:00100000:3.0:1789760748.800716:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 678903457277 time 678914564758 state 0x2 ref 0 00000100:00100000:3.0:1789760748.863747:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503532544 00000100:00100000:3.0:1789760748.863770:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503532544 00000100:00100000:3.0:1789760748.863879:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 678953366095 time 678971551094 state 0x0 ref 1 00000100:00100000:3.0:1789760748.863900:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 678971811510 time 678971551094 state 0x1 ref 1 00000100:00100000:3.0:1789760748.863994:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 678971811510 time 678988600037 state 0x0 ref 1 00000100:00100000:3.0:1789760748.864011:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503532544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760748.864080:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68739072, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x20c88 00010000:00010000:3.0:1789760748.864092:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.864114:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=678s v=(0 0) 00010000:00010000:3.0:1789760748.864119:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.864126:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.864144:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.869975:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68739072 -> 68743168 00002000:00010000:3.0:1789760748.869981:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134280 -> 134288 00010000:00010000:3.0:1789760748.869999:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870014:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870024:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870033:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760748.870040:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870050:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870076:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870086:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed2eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.870276:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503532544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6263us (10765us total) trans 4294984089 rc 0/0 00000100:00100000:3.0:1789760748.870289:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16793 00000100:00100000:3.0:1789760748.870302:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 678971811510 time 678988600037 state 0x2 ref 0 00000100:00100000:3.0:1789760748.919673:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503533056 00000100:00100000:3.0:1789760748.919689:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503533056 00000100:00100000:3.0:1789760748.919764:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679020922171 time 679033448957 state 0x0 ref 1 00000100:00100000:3.0:1789760748.919783:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679033709373 time 679033448957 state 0x1 ref 1 00000100:00100000:3.0:1789760748.919885:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679033709373 time 679044490252 state 0x0 ref 1 00000100:00100000:3.0:1789760748.919900:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503533056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760748.919968:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68755456, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x20ca8 00010000:00010000:3.0:1789760748.919978:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.919998:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760748.920003:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.920012:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.920028:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.926056:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68755456 -> 68759552 00002000:00010000:3.0:1789760748.926062:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134312 -> 134320 00010000:00010000:3.0:1789760748.926079:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.926101:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.926111:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.926120:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760748.926125:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.926136:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.926162:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.926171:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed307 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.926456:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503533056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6490us (6856us total) trans 4294984093 rc 0/0 00000100:00100000:3.0:1789760748.926469:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16797 00000100:00100000:3.0:1789760748.926482:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679033709373 time 679044490252 state 0x2 ref 0 00000100:00100000:3.0:1789760748.961990:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503533568 00000100:00100000:3.0:1789760748.962013:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503533568 00000100:00100000:3.0:1789760748.962091:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679067971288 time 679078551554 state 0x0 ref 1 00000100:00100000:3.0:1789760748.962110:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679078811970 time 679078551554 state 0x1 ref 1 00000100:00100000:3.0:1789760748.962152:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679078811970 time 679086757720 state 0x0 ref 1 00000100:00100000:3.0:1789760748.962167:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503533568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760748.962232:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68771840, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x20cc8 00010000:00010000:3.0:1789760748.962243:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.962262:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760748.962266:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.962274:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.962348:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.967891:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68771840 -> 68775936 00002000:00010000:3.0:1789760748.967897:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134344 -> 134352 00010000:00010000:3.0:1789760748.967913:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.967929:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.967944:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.967952:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760748.967958:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.967968:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.967994:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.968078:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed323 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.968235:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503533568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6063us (6371us total) trans 4294984097 rc 0/0 00000100:00100000:3.0:1789760748.968248:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16801 00000100:00100000:3.0:1789760748.968261:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679078811970 time 679086757720 state 0x2 ref 0 00000001:00080000:3.0:1789760748.997227:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984099 is committed 00000100:00100000:3.0:1789760749.018674:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503534080 00000100:00100000:3.0:1789760749.018688:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503534080 00010000:00010000:3.0:1789760749.075614:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760749.118479:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984106 is committed 00000001:00080000:3.0:1789760749.134650:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984107 is committed 00000001:00080000:3.0:1789760749.155216:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984108 is committed 00000100:00100000:3.0:1789760749.195661:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.219295:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.272746:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760749.298178:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68849664 -> 68853760 00002000:00010000:3.0:1789760749.298193:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134496 -> 134504 00010000:00010000:3.0:1789760749.298213:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ed3a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.298228:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ed3a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.298238:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ed3a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.298246:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760749.298252:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ed3a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.298263:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ed3a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.298288:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ed3a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.298297:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ed3a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.298801:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503536000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13607us (14058us total) trans 4294984116 rc 0/0 00000100:00100000:3.0:1789760749.298825:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16820 00000100:00100000:3.0:1789760749.298839:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679398067243 time 679409761831 state 0x2 ref 0 00000100:00100000:3.0:1789760749.305066:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.335652:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984118 is committed 00000001:00080000:3.0:1789760749.360777:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984119 is committed 00000100:00100000:3.0:1789760749.370955:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.378098:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984120 is committed 00000100:00100000:3.0:1789760749.386002:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.391386:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984121 is committed 00000100:00100000:3.0:1789760749.418662:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.444840:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.452653:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984124 is committed 00000100:00100000:3.0:1789760749.460756:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.477859:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.494212:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.515233:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760749.525337:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68898816 -> 68902912 00002000:00010000:3.0:1789760749.525343:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134592 -> 134600 00010000:00010000:3.0:1789760749.525360:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed3fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.525375:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed3fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.525412:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed3fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.525421:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760749.525426:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed3fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.525437:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed3fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.525460:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed3fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.525674:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed3fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.525866:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503537536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9700us (10727us total) trans 4294984128 rc 0/0 00000100:00100000:3.0:1789760749.525879:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16832 00000100:00100000:3.0:1789760749.525893:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679620827160 time 679640743916 state 0x2 ref 0 00000100:00100000:3.0:1789760749.529891:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.580858:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.603626:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984133 is committed 00000001:00080000:3.0:1789760749.617952:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984134 is committed 00000100:00100000:3.0:1789760749.637344:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503538560 00000100:00100000:3.0:1789760749.637362:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503538560 00000100:00100000:3.0:1789760749.637643:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679736615176 time 679750134556 state 0x0 ref 1 00000100:00100000:3.0:1789760749.637661:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679750394972 time 679750134556 state 0x1 ref 1 00000100:00100000:3.0:1789760749.637730:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679750394972 time 679762336259 state 0x0 ref 1 00000100:00100000:3.0:1789760749.637745:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f70700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503538560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760749.637830:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68931584, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x20e00 00010000:00010000:3.0:1789760749.637842:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.637862:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760749.637867:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.637874:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.637891:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.657278:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68931584 -> 68935680 00002000:00010000:3.0:1789760749.657293:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134656 -> 134664 00010000:00010000:3.0:1789760749.657312:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.657329:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.657340:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.657348:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760749.657354:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.657365:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.657872:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.657887:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed434 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68931584->68935679] (req 68931584->68935679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.658094:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f70700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503538560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 20343us (21075us total) trans 4294984136 rc 0/0 00000100:00100000:3.0:1789760749.658108:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16840 00000100:00100000:3.0:1789760749.658129:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679750394972 time 679762336259 state 0x2 ref 0 00000001:00080000:3.0:1789760749.676306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984137 is committed 00000001:00080000:3.0:1789760749.693202:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984138 is committed 00000001:00080000:3.0:1789760749.714576:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984139 is committed 00000100:00100000:3.0:1789760749.721079:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503539072 00000100:00100000:3.0:1789760749.721096:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503539072 00000100:00100000:3.0:1789760749.721179:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679808822764 time 679824361999 state 0x0 ref 1 00000100:00100000:3.0:1789760749.721198:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679824622415 time 679824361999 state 0x1 ref 1 00000100:00100000:3.0:1789760749.721303:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 679824622415 time 679845907979 state 0x0 ref 1 00000100:00100000:3.0:1789760749.721320:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503539072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760749.721434:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68947968, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x20e20 00010000:00010000:3.0:1789760749.721447:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed450 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.721467:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=679s v=(0 0) 00010000:00010000:3.0:1789760749.721473:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.721481:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68947968->68952063] (req 68947968->68952063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.721499:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68947968->68952063] (req 68947968->68952063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.744193:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984141 is committed 00000001:00080000:3.0:1789760749.764912:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984142 is committed 00000100:00100000:3.0:1789760749.790080:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.798525:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984144 is committed 00000100:00100000:3.0:1789760749.809385:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.860130:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984147 is committed 00000100:00100000:3.0:1789760749.874868:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.883363:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984148 is committed 00000001:00080000:3.0:1789760749.908675:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984150 is committed 00000100:00100000:3.0:1789760749.917137:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760749.917644:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680012304758 time 680025640751 state 0x0 ref 1 00000100:00100000:3.0:1789760749.917660:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680025901167 time 680025640751 state 0x1 ref 1 00000100:00100000:3.0:1789760749.917690:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680025901167 time 680042296333 state 0x0 ref 1 00000100:00100000:3.0:1789760749.917703:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503540480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760749.917968:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68993024, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x20e78 00010000:00010000:3.0:1789760749.917977:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed49d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.917993:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760749.917997:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.918003:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed49d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68993024->68997119] (req 68993024->68997119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.918016:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed49d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68993024->68997119] (req 68993024->68997119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.923935:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984151 is committed 00000100:00100000:3.0:1789760749.933899:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.945333:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984152 is committed 00000100:00100000:3.0:1789760749.970293:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760749.979512:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984154 is committed 00000100:00100000:3.0:1789760749.985617:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760749.993703:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69009408 -> 69013504 00002000:00010000:3.0:1789760749.993710:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134808 -> 134816 00010000:00010000:3.0:1789760749.993723:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083ed4b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69009408->69013503] (req 69009408->69013503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.993737:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083ed4b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69009408->69013503] (req 69009408->69013503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.993746:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083ed4b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69009408->69013503] (req 69009408->69013503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.993753:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760749.993758:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083ed4b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69009408->69013503] (req 69009408->69013503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.993767:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083ed4b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69009408->69013503] (req 69009408->69013503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.993786:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083ed4b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69009408->69013503] (req 69009408->69013503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.993803:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083ed4b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69009408->69013503] (req 69009408->69013503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.994041:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503540992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7866us (8428us total) trans 4294984155 rc 0/0 00000100:00100000:3.0:1789760749.994056:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16859 00000100:00100000:3.0:1789760749.994066:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680095433813 time 680110763066 state 0x2 ref 0 00000100:00100000:3.0:1789760750.000256:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503541120 00000100:00100000:3.0:1789760750.000268:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503541120 00000100:00100000:3.0:1789760750.000636:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680095433813 time 680110763066 state 0x0 ref 1 00000100:00100000:3.0:1789760750.000653:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680111023482 time 680110763066 state 0x1 ref 1 00000100:00100000:3.0:1789760750.000727:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680111023482 time 680125333349 state 0x0 ref 1 00000100:00100000:3.0:1789760750.000741:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503541120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.000789:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69013504, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x20ea0 00010000:00010000:3.0:1789760750.000798:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.000823:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760750.000828:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.000835:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.000850:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.007581:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69013504 -> 69017600 00002000:00010000:3.0:1789760750.007586:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134816 -> 134824 00010000:00010000:3.0:1789760750.007599:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.007611:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.007619:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.007625:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760750.007629:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.007637:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.007654:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.007660:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69013504->69017599] (req 69013504->69017599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.007755:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503541120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7013us (8626us total) trans 4294984156 rc 0/0 00000100:00100000:3.0:1789760750.007764:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16860 00000100:00100000:3.0:1789760750.007773:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680111023482 time 680125333349 state 0x2 ref 0 00000100:00100000:3.0:1789760750.039048:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503541504 00000100:00100000:3.0:1789760750.039059:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503541504 00000100:00100000:3.0:1789760750.039109:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680138686312 time 680148311617 state 0x0 ref 1 00000100:00100000:3.0:1789760750.039123:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680148572033 time 680148311617 state 0x1 ref 1 00000100:00100000:3.0:1789760750.039191:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680148572033 time 680163796765 state 0x0 ref 1 00000100:00100000:3.0:1789760750.039204:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503541504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.039249:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69025792, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x20eb8 00010000:00010000:3.0:1789760750.039257:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.039272:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760750.039275:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.039281:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.039293:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.045874:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69025792 -> 69029888 00002000:00010000:3.0:1789760750.045879:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134840 -> 134848 00010000:00010000:3.0:1789760750.045891:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.045910:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.045919:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.045926:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760750.045930:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.045939:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.045958:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.045966:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed4d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69025792->69029887] (req 69025792->69029887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.046062:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503541504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6855us (7520us total) trans 4294984159 rc 0/0 00000100:00100000:3.0:1789760750.046071:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16863 00000100:00100000:3.0:1789760750.046081:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680148572033 time 680163796765 state 0x2 ref 0 00000100:00100000:3.0:1789760750.054247:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503541632 00000100:00100000:3.0:1789760750.054263:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503541632 00000001:00080000:3.0:1789760750.078052:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984161 is committed 00000001:00080000:3.0:1789760750.094460:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984162 is committed 00000100:00100000:3.0:1789760750.097844:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00010000:00010000:3.0:1789760750.099145:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760750.104964:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69042176 -> 69046272 00002000:00010000:3.0:1789760750.104969:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134872 -> 134880 00010000:00010000:3.0:1789760750.104981:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ed4f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69042176->69046271] (req 69042176->69046271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.104994:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ed4f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69042176->69046271] (req 69042176->69046271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.105004:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ed4f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69042176->69046271] (req 69042176->69046271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.105011:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760750.105016:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ed4f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69042176->69046271] (req 69042176->69046271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.105025:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ed4f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69042176->69046271] (req 69042176->69046271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.105044:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ed4f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69042176->69046271] (req 69042176->69046271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.105052:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ed4f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69042176->69046271] (req 69042176->69046271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.105149:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503542016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6825us (7310us total) trans 4294984163 rc 0/0 00000100:00100000:3.0:1789760750.105159:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16867 00000100:00100000:3.0:1789760750.105170:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680212901915 time 680222915648 state 0x2 ref 0 00000100:00100000:3.0:1789760750.111202:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760750.149138:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760750.165973:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760750.172499:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984167 is committed 00000100:00100000:3.0:1789760750.188882:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760750.194010:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984169 is committed 00000100:00100000:3.0:1789760750.210765:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760750.225655:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503543040 00000100:00100000:3.0:1789760750.225664:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503543040 00000100:00100000:3.0:1789760750.225717:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680313963055 time 680337250393 state 0x0 ref 1 00000100:00100000:3.0:1789760750.225731:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680337510809 time 680337250393 state 0x1 ref 1 00000100:00100000:3.0:1789760750.225796:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680337510809 time 680350401609 state 0x0 ref 1 00000100:00100000:3.0:1789760750.225819:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503543040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.225874:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69074944, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x20f18 00010000:00010000:3.0:1789760750.225883:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.225903:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760750.225908:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.225915:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.225928:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.231316:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69074944 -> 69079040 00002000:00010000:3.0:1789760750.231321:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134936 -> 134944 00010000:00010000:3.0:1789760750.231335:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.231347:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.231356:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.231364:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760750.231368:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.231378:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.231424:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.231432:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed529 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69074944->69079039] (req 69074944->69079039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.231536:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503543040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5706us (5993us total) trans 4294984171 rc 0/0 00000100:00100000:3.0:1789760750.231547:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16875 00000100:00100000:3.0:1789760750.231556:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680337510809 time 680350401609 state 0x2 ref 0 00000100:00100000:3.0:1789760750.234482:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760750.240150:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984172 is committed 00000001:00080000:3.0:1789760750.262097:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984174 is committed 00000100:00100000:3.0:1789760750.265278:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760750.279061:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760750.311372:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503544064 00000100:00100000:3.0:1789760750.312062:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503544064 00000001:00080000:3.0:1789760750.317754:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984179 is committed 00000001:00080000:3.0:1789760750.380968:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984184 is committed 00000001:00080000:3.0:1789760750.403297:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984186 is committed 00000100:00100000:3.0:1789760750.441105:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760750.539159:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503546112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14506us (14913us total) trans 4294984195 rc 0/0 00000100:00100000:3.0:1789760750.539170:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16899 00000100:00100000:3.0:1789760750.539180:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680621601372 time 680649241703 state 0x2 ref 0 00000001:00080000:3.0:1789760750.561637:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984197 is committed 00000001:00080000:3.0:1789760750.571601:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984198 is committed 00000100:00100000:3.0:1789760750.579605:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503546624 00000100:00100000:3.0:1789760750.579615:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503546624 00000100:00100000:3.0:1789760750.579669:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680682151735 time 680691730706 state 0x0 ref 1 00000100:00100000:3.0:1789760750.579682:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680691991122 time 680691730706 state 0x1 ref 1 00000100:00100000:3.0:1789760750.579811:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680691991122 time 680704408189 state 0x0 ref 1 00000100:00100000:3.0:1789760750.579826:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503546624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.579882:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69189632, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x20ff8 00010000:00010000:3.0:1789760750.579892:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.579906:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760750.579911:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.579916:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.579929:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.584579:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69189632 -> 69193728 00002000:00010000:3.0:1789760750.584585:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 135160 -> 135168 00010000:00010000:3.0:1789760750.584599:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.584612:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.584621:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.584629:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760750.584634:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.584644:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.584663:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.584671:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed5ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69189632->69193727] (req 69189632->69193727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.584888:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503546624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5059us (5375us total) trans 4294984199 rc 0/0 00000100:00100000:3.0:1789760750.584900:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16903 00000100:00100000:3.0:1789760750.584909:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680691991122 time 680704408189 state 0x2 ref 0 00000100:00100000:3.0:1789760750.622025:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503547136 00000100:00100000:3.0:1789760750.622036:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503547136 00000100:00100000:3.0:1789760750.622089:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680723685895 time 680734697393 state 0x0 ref 1 00000100:00100000:3.0:1789760750.622104:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680734957809 time 680734697393 state 0x1 ref 1 00000100:00100000:3.0:1789760750.622133:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680734957809 time 680746739984 state 0x0 ref 1 00000100:00100000:3.0:1789760750.622146:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503547136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.622296:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69206016, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x21018 00010000:00010000:3.0:1789760750.622306:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed609 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.622321:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760750.622325:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.622332:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed609 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69206016->69210111] (req 69206016->69210111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.622346:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed609 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69206016->69210111] (req 69206016->69210111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.800323:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503548928 00000100:00100000:3.0:1789760750.800333:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503548928 00000100:00100000:3.0:1789760750.800386:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680904202065 time 680915391061 state 0x0 ref 1 00000100:00100000:3.0:1789760750.800419:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680915651477 time 680915391061 state 0x1 ref 1 00000100:00100000:3.0:1789760750.800444:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680915651477 time 680925050761 state 0x0 ref 1 00000100:00100000:3.0:1789760750.800460:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503548928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.800507:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69263360, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x21088 00010000:00010000:3.0:1789760750.800516:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed66b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.800540:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760750.800545:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.800551:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed66b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69263360->69267455] (req 69263360->69267455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.800566:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed66b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69263360->69267455] (req 69263360->69267455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.866315:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503549440 00000100:00100000:3.0:1789760750.866325:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503549440 00000100:00100000:3.0:1789760750.866381:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680949226564 time 680968158346 state 0x0 ref 1 00000100:00100000:3.0:1789760750.866431:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680968418762 time 680968158346 state 0x1 ref 1 00000100:00100000:3.0:1789760750.866463:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680968418762 time 680991069926 state 0x0 ref 1 00000100:00100000:3.0:1789760750.866475:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503549440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.866522:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69279744, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x210a8 00010000:00010000:3.0:1789760750.866539:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.866561:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=680s v=(0 0) 00010000:00010000:3.0:1789760750.866566:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.866571:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.866593:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.874684:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69279744 -> 69283840 00002000:00010000:3.0:1789760750.874690:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 135336 -> 135344 00010000:00010000:3.0:1789760750.874702:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.874715:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.874724:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.874732:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760750.874737:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.874746:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.874771:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.874778:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed687 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69279744->69283839] (req 69279744->69283839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.874987:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503549440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8508us (8992us total) trans 4294984221 rc 0/0 00000100:00100000:3.0:1789760750.874997:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16925 00000100:00100000:3.0:1789760750.875007:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 680968418762 time 680991069926 state 0x2 ref 0 00000100:00100000:3.0:1789760750.921514:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503549952 00000100:00100000:3.0:1789760750.921525:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503549952 00000100:00100000:3.0:1789760750.921580:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 681015484194 time 681031681203 state 0x0 ref 1 00000100:00100000:3.0:1789760750.921672:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 681031941619 time 681031681203 state 0x1 ref 1 00000100:00100000:3.0:1789760750.921695:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 681031941619 time 681046302041 state 0x0 ref 1 00000100:00100000:3.0:1789760750.921709:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503549952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760750.921755:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69296128, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x210c8 00010000:00010000:3.0:1789760750.921765:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed6a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.921781:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760750.921786:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.921793:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed6a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69296128->69300223] (req 69296128->69300223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.921818:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ed6a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69296128->69300223] (req 69296128->69300223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.955902:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984227 is committed 00000001:00080000:3.0:1789760750.968425:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984228 is committed 00000001:00080000:3.0:1789760750.980244:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984229 is committed 00000001:00080000:3.0:1789760751.022083:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984232 is committed 00000001:00080000:3.0:1789760751.034451:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984233 is committed 00000001:00080000:3.0:1789760751.086743:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984236 is committed 00000001:00080000:3.0:1789760751.104211:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984237 is committed 00000001:00080000:3.0:1789760751.121242:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984238 is committed 00010000:00010000:3.0:1789760751.122888:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760751.132066:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984239 is committed 00000001:00080000:3.0:1789760751.144875:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984240 is committed 00000001:00080000:3.0:1789760751.160161:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984241 is committed 00000001:00080000:3.0:1789760751.173654:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984242 is committed 00000001:00080000:3.0:1789760751.188634:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984243 is committed 00000001:00080000:3.0:1789760751.202350:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984244 is committed 00000001:00080000:3.0:1789760751.218951:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984245 is committed 00000001:00080000:3.0:1789760751.230341:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984246 is committed 00000001:00080000:3.0:1789760751.247539:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984247 is committed 00000001:00080000:3.0:1789760751.272883:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984249 is committed 00000001:00080000:3.0:1789760751.301175:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984250 is committed 00000100:00100000:3.0:1789760751.316993:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523596928 00000100:00100000:3.0:1789760751.317002:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523596928 00000100:00100000:3.0:1789760751.317054:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:3.0:1789760751.317066:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523596928:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760751.317110:0:6517:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:3.0:1789760751.317165:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523596928:12345-0@lo:41:osp-pre-0-1.0 Request processed in 101us (542us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760751.317173:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000001:00080000:3.0:1789760751.386705:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984255 is committed 00000001:00080000:3.0:1789760751.405357:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984256 is committed 00000001:00080000:3.0:1789760751.426926:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984257 is committed 00000001:00080000:3.0:1789760751.449687:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984258 is committed 00000001:00080000:3.0:1789760751.464729:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984259 is committed 00000100:00100000:3.0:1789760751.543376:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.588485:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.611615:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.693181:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503555968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8872us (9115us total) trans 4294984272 rc 0/0 00000100:00100000:3.0:1789760751.693192:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16976 00000100:00100000:3.0:1789760751.693202:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 681794193710 time 681808898666 state 0x2 ref 0 00000100:00100000:3.0:1789760751.742222:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503556480 00000100:00100000:3.0:1789760751.742233:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503556480 00000100:00100000:3.0:1789760751.742290:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 681832933906 time 681852812505 state 0x0 ref 1 00000100:00100000:3.0:1789760751.742306:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 681853072921 time 681852812505 state 0x1 ref 1 00000100:00100000:3.0:1789760751.742339:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 681853072921 time 681866945330 state 0x0 ref 1 00000100:00100000:3.0:1789760751.742351:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503556480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760751.742442:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69505024, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x21260 00010000:00010000:3.0:1789760751.742452:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed808 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.742469:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=681s v=(0 0) 00010000:00010000:3.0:1789760751.742473:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.742480:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed808 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69505024->69509119] (req 69505024->69509119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.742494:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ed808 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69505024->69509119] (req 69505024->69509119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.780569:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.794587:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.804199:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.812901:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760751.819087:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984282 is committed 00000100:00100000:3.0:1789760751.825184:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.840830:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.853580:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.868066:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.905499:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760751.964848:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760752.018918:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760752.028133:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69578752 -> 69582848 00002000:00010000:3.0:1789760752.028148:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 135920 -> 135928 00010000:00010000:3.0:1789760752.028167:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ed886 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69578752->69582847] (req 69578752->69582847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.028181:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ed886 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69578752->69582847] (req 69578752->69582847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.028191:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ed886 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69578752->69582847] (req 69578752->69582847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.028198:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.028203:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ed886 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69578752->69582847] (req 69578752->69582847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.028212:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ed886 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69578752->69582847] (req 69578752->69582847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.028231:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ed886 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69578752->69582847] (req 69578752->69582847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.028240:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ed886 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69578752->69582847] (req 69578752->69582847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.028334:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503558784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8823us (9421us total) trans 4294984294 rc 0/0 00000100:00100000:3.0:1789760752.028344:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 16998 00000100:00100000:3.0:1789760752.028354:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682125208764 time 682144098573 state 0x2 ref 0 00000100:00100000:3.0:1789760752.092000:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503559936 00000100:00100000:3.0:1789760752.092010:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503559936 00000100:00100000:3.0:1789760752.092062:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682177245238 time 682201878867 state 0x0 ref 1 00000100:00100000:3.0:1789760752.092100:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682202139283 time 682201878867 state 0x1 ref 1 00000100:00100000:3.0:1789760752.092131:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682202139283 time 682216738091 state 0x0 ref 1 00000100:00100000:3.0:1789760752.092143:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503559936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.092187:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69595136, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21310 00010000:00010000:3.0:1789760752.092197:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.092212:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.092216:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.092222:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.092249:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.101076:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69595136 -> 69599232 00002000:00010000:3.0:1789760752.101085:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 135952 -> 135960 00010000:00010000:3.0:1789760752.101097:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.101110:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.101126:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.101133:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.101139:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.101148:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.101167:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.101175:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed8a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69595136->69599231] (req 69595136->69599231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.101365:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503559936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9218us (9503us total) trans 4294984298 rc 0/0 00000100:00100000:3.0:1789760752.101375:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17002 00000100:00100000:3.0:1789760752.101384:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682202139283 time 682216738091 state 0x2 ref 0 00000100:00100000:3.0:1789760752.107896:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760752.144751:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69607424 -> 69611520 00002000:00010000:3.0:1789760752.144758:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 135976 -> 135984 00010000:00010000:3.0:1789760752.144771:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ed8b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69607424->69611519] (req 69607424->69611519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.144785:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ed8b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69607424->69611519] (req 69607424->69611519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.144806:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ed8b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69607424->69611519] (req 69607424->69611519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.144814:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.144819:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ed8b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69607424->69611519] (req 69607424->69611519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.144829:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ed8b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69607424->69611519] (req 69607424->69611519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.144850:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ed8b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69607424->69611519] (req 69607424->69611519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.144859:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ed8b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69607424->69611519] (req 69607424->69611519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.145115:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503560320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7893us (8221us total) trans 4294984301 rc 0/0 00000100:00100000:3.0:1789760752.145126:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17005 00000100:00100000:3.0:1789760752.145135:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682248284972 time 682261812182 state 0x2 ref 0 00010000:00010000:3.0:1789760752.155479:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760752.159675:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760752.231995:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760752.242991:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69623808 -> 69627904 00002000:00010000:3.0:1789760752.242996:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136008 -> 136016 00010000:00010000:3.0:1789760752.243011:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ed8d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69623808->69627903] (req 69623808->69627903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.243023:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ed8d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69623808->69627903] (req 69623808->69627903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.243033:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ed8d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69623808->69627903] (req 69623808->69627903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.243041:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.243045:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ed8d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69623808->69627903] (req 69623808->69627903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.243055:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ed8d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69623808->69627903] (req 69623808->69627903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.243076:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ed8d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69623808->69627903] (req 69623808->69627903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.243084:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ed8d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69623808->69627903] (req 69623808->69627903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.243558:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503560832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7004us (11567us total) trans 4294984305 rc 0/0 00000100:00100000:3.0:1789760752.243569:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17009 00000100:00100000:3.0:1789760752.243579:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682340305870 time 682361141410 state 0x2 ref 0 00000100:00100000:3.0:1789760752.249634:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760752.291497:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760752.325072:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760752.329858:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682416586810 time 682454457125 state 0x0 ref 1 00000100:00100000:3.0:1789760752.329883:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503561344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.329940:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69640192, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21368 00010000:00010000:3.0:1789760752.329950:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.329966:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.329971:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.329978:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.329991:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.338607:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69640192 -> 69644288 00002000:00010000:3.0:1789760752.338612:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136040 -> 136048 00010000:00010000:3.0:1789760752.338625:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.338641:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.338650:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.338658:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.338663:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.338672:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.338690:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.338697:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69640192->69644287] (req 69640192->69644287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.339000:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503561344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9120us (13931us total) trans 4294984309 rc 0/0 00000100:00100000:3.0:1789760752.339012:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17013 00000100:00100000:3.0:1789760752.339021:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682416586810 time 682454457125 state 0x2 ref 0 00000100:00100000:3.0:1789760752.353010:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503561472 00000100:00100000:3.0:1789760752.353026:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503561472 00000100:00100000:3.0:1789760752.353105:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682416586810 time 682454457125 state 0x0 ref 1 00000100:00100000:3.0:1789760752.353123:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682454717541 time 682454457125 state 0x1 ref 1 00000100:00100000:3.0:1789760752.353165:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682454717541 time 682477771347 state 0x0 ref 1 00000100:00100000:3.0:1789760752.353179:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503561472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.353243:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69644288, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21370 00010000:00010000:3.0:1789760752.353254:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.353282:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.353288:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.353295:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.353312:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.367728:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69644288 -> 69648384 00002000:00010000:3.0:1789760752.367735:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136048 -> 136056 00010000:00010000:3.0:1789760752.367751:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.367766:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.367780:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.367798:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.367805:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.367815:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.367840:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.367848:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed8f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69644288->69648383] (req 69644288->69648383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.368314:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503561472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15127us (15676us total) trans 4294984310 rc 0/0 00000100:00100000:3.0:1789760752.368328:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17014 00000100:00100000:3.0:1789760752.368340:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682454717541 time 682477771347 state 0x2 ref 0 00000100:00100000:3.0:1789760752.399077:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503561856 00000100:00100000:3.0:1789760752.399089:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503561856 00000100:00100000:3.0:1789760752.399165:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682501210620 time 682513413759 state 0x0 ref 1 00000100:00100000:3.0:1789760752.399186:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682513674175 time 682513413759 state 0x1 ref 1 00000100:00100000:3.0:1789760752.411386:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503561984 00000100:00100000:3.0:1789760752.411442:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503561984 00000100:00100000:3.0:1789760752.411530:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682513674175 time 682524107958 state 0x0 ref 1 00000100:00100000:3.0:1789760752.411550:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682524368374 time 682524107958 state 0x1 ref 1 00000100:00100000:3.0:1789760752.411577:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682524368374 time 682536183467 state 0x0 ref 1 00000100:00100000:3.0:1789760752.411592:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503561984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.411656:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69660672, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21390 00010000:00010000:3.0:1789760752.411669:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed912 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.411689:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.411694:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.411702:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69660672->69664767] (req 69660672->69664767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.411719:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ed912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69660672->69664767] (req 69660672->69664767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.423343:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503562112 00000100:00100000:3.0:1789760752.423356:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503562112 00000100:00100000:3.0:1789760752.423462:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682524368374 time 682536183467 state 0x0 ref 1 00000100:00100000:3.0:1789760752.423481:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682536443883 time 682536183467 state 0x1 ref 1 00000100:00100000:3.0:1789760752.423695:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682536443883 time 682548299191 state 0x0 ref 1 00000100:00100000:3.0:1789760752.423709:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503562112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.423774:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69664768, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21398 00010000:00010000:3.0:1789760752.423785:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed919 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.423813:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.423818:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.423890:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed919 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69664768->69668863] (req 69664768->69668863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.423910:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ed919 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69664768->69668863] (req 69664768->69668863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.430245:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984315 is committed 00000001:00080000:3.0:1789760752.449970:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984316 is committed 00000100:00100000:3.0:1789760752.469079:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760752.481513:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984318 is committed 00000100:00100000:3.0:1789760752.521313:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760752.536829:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503563008 00000100:00100000:3.0:1789760752.536844:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503563008 00000100:00100000:3.0:1789760752.537084:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682632465105 time 682646364458 state 0x0 ref 1 00000100:00100000:3.0:1789760752.537121:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682646624874 time 682646364458 state 0x1 ref 1 00000100:00100000:3.0:1789760752.537156:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682646624874 time 682661762607 state 0x0 ref 1 00000100:00100000:3.0:1789760752.537172:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503563008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.537240:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69693440, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x213d0 00010000:00010000:3.0:1789760752.537251:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.537272:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.537277:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.537284:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.537302:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.546521:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69693440 -> 69697536 00002000:00010000:3.0:1789760752.546532:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136144 -> 136152 00010000:00010000:3.0:1789760752.546551:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.546567:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.546589:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.546599:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.546605:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.546616:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.546641:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.546651:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed94a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69693440->69697535] (req 69693440->69697535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.546808:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503563008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9632us (10124us total) trans 4294984322 rc 0/0 00000100:00100000:3.0:1789760752.546821:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17026 00000100:00100000:3.0:1789760752.546835:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682646624874 time 682661762607 state 0x2 ref 0 00000100:00100000:3.0:1789760752.616451:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503563520 00000100:00100000:3.0:1789760752.616467:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503563520 00000100:00100000:3.0:1789760752.616566:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682696665559 time 682720435813 state 0x0 ref 1 00000100:00100000:3.0:1789760752.616585:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682720696229 time 682720435813 state 0x1 ref 1 00000100:00100000:3.0:1789760752.616625:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682720696229 time 682741230910 state 0x0 ref 1 00000100:00100000:3.0:1789760752.616639:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503563520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.616702:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69709824, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x213f0 00010000:00010000:3.0:1789760752.616716:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.616735:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.616740:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.616747:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.616763:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.628580:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69709824 -> 69713920 00002000:00010000:3.0:1789760752.628587:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136176 -> 136184 00010000:00010000:3.0:1789760752.628603:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.628619:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.628630:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.628638:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.628644:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.628654:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.628680:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.628690:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed966 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69709824->69713919] (req 69709824->69713919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.628816:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503563520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12172us (12763us total) trans 4294984326 rc 0/0 00000100:00100000:3.0:1789760752.628838:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17030 00000100:00100000:3.0:1789760752.628851:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682720696229 time 682741230910 state 0x2 ref 0 00002000:00010000:3.0:1789760752.658713:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69713920 -> 69718016 00002000:00010000:3.0:1789760752.658726:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136184 -> 136192 00010000:00010000:3.0:1789760752.658743:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed96d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69713920->69718015] (req 69713920->69718015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.658758:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed96d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69713920->69718015] (req 69713920->69718015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.658769:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed96d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69713920->69718015] (req 69713920->69718015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.658777:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.658782:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed96d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69713920->69718015] (req 69713920->69718015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.659064:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed96d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69713920->69718015] (req 69713920->69718015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.659088:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed96d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69713920->69718015] (req 69713920->69718015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.659096:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ed96d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69713920->69718015] (req 69713920->69718015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.659278:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503563648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 20684us (21143us total) trans 4294984327 rc 0/0 00000100:00100000:3.0:1789760752.659289:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17031 00000100:00100000:3.0:1789760752.659301:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682741491326 time 682763181186 state 0x2 ref 0 00000001:00080000:3.0:1789760752.674179:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984328 is committed 00000001:00080000:3.0:1789760752.690094:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984329 is committed 00000100:00100000:3.0:1789760752.697216:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503564032 00000100:00100000:3.0:1789760752.697229:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503564032 00000100:00100000:3.0:1789760752.697310:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682789965443 time 682807203919 state 0x0 ref 1 00000100:00100000:3.0:1789760752.697329:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682807464335 time 682807203919 state 0x1 ref 1 00000100:00100000:3.0:1789760752.697448:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682807464335 time 682822053461 state 0x0 ref 1 00000100:00100000:3.0:1789760752.697463:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503564032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.697539:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69726208, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21410 00010000:00010000:3.0:1789760752.697550:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed982 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.697570:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.697575:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.697583:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69726208->69730303] (req 69726208->69730303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.697599:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ed982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69726208->69730303] (req 69726208->69730303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.703164:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984330 is committed 00000100:00100000:3.0:1789760752.711525:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503564160 00000100:00100000:3.0:1789760752.711539:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503564160 00000100:00100000:3.0:1789760752.711612:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682807464335 time 682822053461 state 0x0 ref 1 00000100:00100000:3.0:1789760752.711631:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682822313877 time 682822053461 state 0x1 ref 1 00000100:00100000:3.0:1789760752.711667:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682822313877 time 682836273069 state 0x0 ref 1 00000100:00100000:3.0:1789760752.711681:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503564160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.711747:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69730304, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21418 00010000:00010000:3.0:1789760752.711758:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.711797:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.711804:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.711812:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.711840:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.719366:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69730304 -> 69734400 00002000:00010000:3.0:1789760752.719373:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136216 -> 136224 00010000:00010000:3.0:1789760752.719429:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719445:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719456:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719465:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.719472:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719482:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719741:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719751:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed989 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69730304->69734399] (req 69730304->69734399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.720038:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503564160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8351us (8665us total) trans 4294984331 rc 0/0 00000100:00100000:3.0:1789760752.720051:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17035 00000100:00100000:3.0:1789760752.720063:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682822313877 time 682836273069 state 0x2 ref 0 00000100:00100000:3.0:1789760752.775735:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503564672 00000100:00100000:3.0:1789760752.775748:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503564672 00000100:00100000:3.0:1789760752.775864:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682866935460 time 682880360112 state 0x0 ref 1 00000100:00100000:3.0:1789760752.775882:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682880620528 time 682880360112 state 0x1 ref 1 00000100:00100000:3.0:1789760752.775912:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682880620528 time 682900518071 state 0x0 ref 1 00000100:00100000:3.0:1789760752.775925:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503564672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.775991:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69746688, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x21438 00010000:00010000:3.0:1789760752.776003:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.776023:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.776030:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.776038:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.776054:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.787798:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69746688 -> 69750784 00002000:00010000:3.0:1789760752.787804:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760751 -> 1789760752 00002000:00010000:3.0:1789760752.787806:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760751 -> 1789760752 00002000:00010000:3.0:1789760752.787810:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136248 -> 136256 00010000:00010000:3.0:1789760752.787896:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.787913:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.787924:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.787932:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.787938:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.787949:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.787980:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.787988:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69746688->69750783] (req 69746688->69750783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.788181:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503564672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12249us (12552us total) trans 4294984335 rc 0/0 00000100:00100000:3.0:1789760752.788194:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17039 00000100:00100000:3.0:1789760752.788207:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682880620528 time 682900518071 state 0x2 ref 0 00000100:00100000:3.0:1789760752.841149:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503565184 00000100:00100000:3.0:1789760752.841163:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503565184 00000100:00100000:3.0:1789760752.841295:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682938928222 time 682950077049 state 0x0 ref 1 00000100:00100000:3.0:1789760752.841317:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682950337465 time 682950077049 state 0x1 ref 1 00000100:00100000:3.0:1789760752.841451:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682950337465 time 682966055956 state 0x0 ref 1 00000100:00100000:3.0:1789760752.841467:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503565184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.841544:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69763072, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x21458 00010000:00010000:3.0:1789760752.841557:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.841575:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=682s v=(0 0) 00010000:00010000:3.0:1789760752.841582:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.841590:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69763072->69767167] (req 69763072->69767167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.841605:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69763072->69767167] (req 69763072->69767167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.850422:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984339 is committed 00000001:00080000:3.0:1789760752.863561:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984340 is committed 00000100:00100000:3.0:1789760752.920322:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503565696 00000100:00100000:3.0:1789760752.920336:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503565696 00000100:00100000:3.0:1789760752.920494:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 682994083908 time 683021932276 state 0x0 ref 1 00000100:00100000:3.0:1789760752.920572:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683022192692 time 683021932276 state 0x1 ref 1 00000100:00100000:3.0:1789760752.920667:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683022192692 time 683045272108 state 0x0 ref 1 00000100:00100000:3.0:1789760752.920682:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503565696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760752.920745:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69779456, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x21478 00010000:00010000:3.0:1789760752.920757:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.920777:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760752.920790:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.920799:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.920817:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.934883:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69779456 -> 69783552 00002000:00010000:3.0:1789760752.934894:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136312 -> 136320 00010000:00010000:3.0:1789760752.934908:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.934920:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.934929:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.934936:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760752.934941:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.934950:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.934972:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.934980:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69779456->69783551] (req 69779456->69783551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.935100:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503565696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14414us (14973us total) trans 4294984343 rc 0/0 00000100:00100000:3.0:1789760752.935110:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17047 00000100:00100000:3.0:1789760752.935119:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683022192692 time 683045272108 state 0x2 ref 0 00000001:00080000:3.0:1789760752.957944:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984344 is committed 00000001:00080000:3.0:1789760753.003485:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984346 is committed 00000100:00100000:3.0:1789760753.018980:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503566208 00000100:00100000:3.0:1789760753.019004:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503566208 00000100:00100000:3.0:1789760753.019099:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683087257555 time 683116583858 state 0x0 ref 1 00000100:00100000:3.0:1789760753.019119:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683116844274 time 683116583858 state 0x1 ref 1 00000100:00100000:3.0:1789760753.019164:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683116844274 time 683143770320 state 0x0 ref 1 00000100:00100000:3.0:1789760753.019180:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503566208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.019241:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69795840, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x21498 00010000:00010000:3.0:1789760753.019252:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.019414:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.019420:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.019428:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.019445:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.031016:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69795840 -> 69799936 00002000:00010000:3.0:1789760753.031022:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136344 -> 136352 00010000:00010000:3.0:1789760753.031040:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.031055:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.031065:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.031073:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.031080:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.031090:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.031115:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.031123:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ed9f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69795840->69799935] (req 69795840->69799935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.031273:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503566208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12089us (15091us total) trans 4294984347 rc 0/0 00000100:00100000:3.0:1789760753.031287:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17051 00000100:00100000:3.0:1789760753.031299:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683116844274 time 683143770320 state 0x2 ref 0 00000001:00080000:3.0:1789760753.081991:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984350 is committed 00000100:00100000:3.0:1789760753.094505:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503566720 00000100:00100000:3.0:1789760753.094529:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503566720 00000100:00100000:3.0:1789760753.094608:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683183582981 time 683199038533 state 0x0 ref 1 00000100:00100000:3.0:1789760753.094628:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683199298949 time 683199038533 state 0x1 ref 1 00000100:00100000:3.0:1789760753.094665:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683199298949 time 683219271300 state 0x0 ref 1 00000100:00100000:3.0:1789760753.094680:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503566720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.094745:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69812224, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x214b8 00010000:00010000:3.0:1789760753.094757:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.094788:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.094794:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.094801:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.094819:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.102433:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69812224 -> 69816320 00002000:00010000:3.0:1789760753.102443:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136376 -> 136384 00010000:00010000:3.0:1789760753.102460:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.102475:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.102485:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.102493:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.102615:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.102626:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.102651:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.102660:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69812224->69816319] (req 69812224->69816319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.102863:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503566720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8178us (8525us total) trans 4294984351 rc 0/0 00000100:00100000:3.0:1789760753.102875:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17055 00000100:00100000:3.0:1789760753.102887:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683199298949 time 683219271300 state 0x2 ref 0 00000001:00080000:3.0:1789760753.124162:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984352 is committed 00000001:00080000:3.0:1789760753.143459:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984353 is committed 00000100:00100000:3.0:1789760753.149795:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503567104 00000100:00100000:3.0:1789760753.149809:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503567104 00000100:00100000:3.0:1789760753.149891:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683240022535 time 683255735854 state 0x0 ref 1 00000100:00100000:3.0:1789760753.149912:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683255996270 time 683255735854 state 0x1 ref 1 00000100:00100000:3.0:1789760753.149948:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683255996270 time 683274553232 state 0x0 ref 1 00000100:00100000:3.0:1789760753.149963:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503567104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.150033:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69824512, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x214d0 00010000:00010000:3.0:1789760753.150047:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.150067:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.150073:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.150080:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69824512->69828607] (req 69824512->69828607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.150097:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eda2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69824512->69828607] (req 69824512->69828607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.156551:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984354 is committed 00000100:00100000:3.0:1789760753.162697:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503567232 00000100:00100000:3.0:1789760753.162709:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503567232 00000100:00100000:3.0:1789760753.162791:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683255996270 time 683274553232 state 0x0 ref 1 00000100:00100000:3.0:1789760753.162811:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683274813648 time 683274553232 state 0x1 ref 1 00000100:00100000:3.0:1789760753.162853:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683274813648 time 683287459456 state 0x0 ref 1 00000100:00100000:3.0:1789760753.162868:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503567232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.162941:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69828608, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x214d8 00010000:00010000:3.0:1789760753.162954:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.162975:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.162979:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.162987:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.163004:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.170577:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760753.171807:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69828608 -> 69832704 00002000:00010000:3.0:1789760753.171814:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136408 -> 136416 00010000:00010000:3.0:1789760753.171837:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.171852:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.171866:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.171873:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.171880:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.171898:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.171923:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.171933:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69828608->69832703] (req 69828608->69832703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.172053:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503567232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9180us (9502us total) trans 4294984355 rc 0/0 00000100:00100000:3.0:1789760753.172065:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17059 00000100:00100000:3.0:1789760753.172076:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683274813648 time 683287459456 state 0x2 ref 0 00000001:00080000:3.0:1789760753.192667:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984356 is committed 00000100:00100000:3.0:1789760753.225277:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503567616 00000100:00100000:3.0:1789760753.225298:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503567616 00000100:00100000:3.0:1789760753.312148:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503568256 00000100:00100000:3.0:1789760753.312162:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503568256 00000100:00100000:3.0:1789760753.312241:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683403123536 time 683422565048 state 0x0 ref 1 00000100:00100000:3.0:1789760753.312259:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683422825464 time 683422565048 state 0x1 ref 1 00000100:00100000:3.0:1789760753.312298:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683422825464 time 683436904091 state 0x0 ref 1 00000100:00100000:3.0:1789760753.312311:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503568256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.312375:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69861376, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x21518 00010000:00010000:3.0:1789760753.312386:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.312452:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.312457:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.312466:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.312483:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.320169:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69861376 -> 69865472 00002000:00010000:3.0:1789760753.320176:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136472 -> 136480 00010000:00010000:3.0:1789760753.320193:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.320209:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.320225:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.320233:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.320239:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.320249:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.320276:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.320285:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eda69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69861376->69865471] (req 69861376->69865471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.320526:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503568256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8209us (8735us total) trans 4294984363 rc 0/0 00000100:00100000:3.0:1789760753.320539:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17067 00000100:00100000:3.0:1789760753.320552:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683422825464 time 683436904091 state 0x2 ref 0 00000100:00100000:3.0:1789760753.366271:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc95f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523597184:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760753.366359:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.366604:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc95f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:afc717a1-11ba-4ef0-ab10-103aa692ca98:3639:1876699523597312:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.366653:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.366682:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc8feaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523597824:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.366722:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.366751:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc8febb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523598336:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.366790:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.366819:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789760753.367340:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc95f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3639:1876699523597696:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.367377:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.367528:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc8feb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523598208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.367558:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760753.367599:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523597312 00000100:00100000:3.0:1789760753.367605:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523597312 00000100:00100000:3.0:1789760753.367650:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789760753.367659:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dea8a8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3639:x1876699523597312:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.367704:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dea8a8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3639:x1876699523597312:12345-0@lo:400:kworker.0 Request processed in 46us (1049us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.367711:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789760753.367751:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523597824 00000100:00100000:3.0:1789760753.367756:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523597824 00000100:00100000:3.0:1789760753.367798:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:3.0:1789760753.367807:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523597824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.367844:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523597824:12345-0@lo:400:kworker.0 Request processed in 39us (1123us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.367849:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:3.0:1789760753.367867:0:8431:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523598336 00000100:00100000:3.0:1789760753.367871:0:8431:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523598336 00000100:00100000:3.0:1789760753.367897:0:8431:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523598208 00000100:00100000:3.0:1789760753.367900:0:8431:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523598208 00000100:00100000:3.0:1789760753.367916:0:8431:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1789760753.367922:0:8431:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523598336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.367950:0:8431:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523598336:12345-0@lo:400:kworker.0 Request processed in 29us (1171us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.367955:0:8431:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1789760753.367966:0:8431:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789760753.367971:0:8431:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523598208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.367992:0:8431:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523598208:12345-0@lo:400:kworker.0 Request processed in 22us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.367997:0:8431:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789760753.369188:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc95f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523597184:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760753.369245:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8feb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523598208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.369259:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc95f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:afc717a1-11ba-4ef0-ab10-103aa692ca98:3639:1876699523597312:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.369271:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8feaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523597824:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.369283:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8febb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523598336:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.371380:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523597568 00000100:00100000:3.0:1789760753.371408:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523597568 00000100:00100000:3.0:1789760753.371474:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:3.0:1789760753.371487:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523597568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.372037:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523597568:12345-0@lo:400:kworker.0 Request processed in 553us (927us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.372045:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000001:00080000:3.0:1789760753.373204:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984366 is committed 00002000:00010000:3.0:1789760753.373293:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69873664 -> 69877760 00002000:00010000:3.0:1789760753.373298:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136496 -> 136504 00010000:00010000:3.0:1789760753.373315:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eda7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69873664->69877759] (req 69873664->69877759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.373331:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eda7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69873664->69877759] (req 69873664->69877759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.373342:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eda7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69873664->69877759] (req 69873664->69877759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.373350:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.373355:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eda7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69873664->69877759] (req 69873664->69877759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.373367:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eda7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69873664->69877759] (req 69873664->69877759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.373411:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eda7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69873664->69877759] (req 69873664->69877759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.373420:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eda7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69873664->69877759] (req 69873664->69877759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.373519:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503568640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15611us (15992us total) trans 4294984366 rc 0/0 00000100:00100000:3.0:1789760753.373545:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17070 00000100:00100000:3.0:1789760753.373557:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683466842702 time 683482497463 state 0x2 ref 0 00000100:00100000:3.0:1789760753.379881:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 916 00000100:00100000:3.0:1789760753.379917:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3636:x1876699523597440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.380047:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3636:x1876699523597440:12345-0@lo:400:kworker.0 Request processed in 148us (9592us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.380056:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 916 00000100:00100000:3.0:1789760753.380094:0:8237:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523598464 00000100:00100000:3.0:1789760753.380101:0:8237:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523598464 00000100:00100000:3.0:1789760753.380143:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 917 00000100:00100000:3.0:1789760753.380149:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3636:x1876699523598464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.380180:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3636:x1876699523598464:12345-0@lo:400:kworker.0 Request processed in 32us (2344us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.380186:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 917 00000100:00100000:3.0:1789760753.380303:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc95f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3639:1876699523597696:0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.384039:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760753.384566:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69877760, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x21538 00010000:00010000:3.0:1789760753.384579:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.384600:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.384606:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.384614:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.384636:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.392109:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523598080 00000100:00100000:3.0:1789760753.392120:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523598080 00000100:00100000:3.0:1789760753.392197:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523598592 00000100:00100000:3.0:1789760753.392201:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523598592 00000100:00100000:3.0:1789760753.392229:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 918 00000100:00100000:3.0:1789760753.392240:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3637:x1876699523598080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.392333:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3637:x1876699523598080:12345-0@lo:400:kworker.0 Request processed in 94us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.392341:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 918 00000100:00100000:3.0:1789760753.392365:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 919 00000100:00100000:3.0:1789760753.392376:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3637:x1876699523598592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760753.398907:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3637:x1876699523598592:12345-0@lo:400:kworker.0 Request processed in 6519us (6918us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760753.398925:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 919 00002000:00010000:3.0:1789760753.413148:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69877760 -> 69881856 00002000:00010000:3.0:1789760753.413155:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136504 -> 136512 00010000:00010000:3.0:1789760753.413173:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.413188:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.413202:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.413209:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.413215:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.413225:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.413250:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.413259:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69877760->69881855] (req 69877760->69881855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.413445:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503568768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 29075us (29360us total) trans 4294984367 rc 0/0 00000100:00100000:3.0:1789760753.413457:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17071 00000100:00100000:3.0:1789760753.413480:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683482757879 time 683508904987 state 0x2 ref 0 00000100:00100000:3.0:1789760753.423041:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760753.443716:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503569024 00000100:00100000:3.0:1789760753.443730:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503569024 00000100:00100000:3.0:1789760753.443818:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683509165403 time 683547895710 state 0x0 ref 1 00000100:00100000:3.0:1789760753.443971:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683548156126 time 683547895710 state 0x1 ref 1 00000100:00100000:3.0:1789760753.444013:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683548156126 time 683568619917 state 0x0 ref 1 00000100:00100000:3.0:1789760753.444028:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503569024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.444093:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69885952, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x21548 00010000:00010000:3.0:1789760753.444103:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.444124:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.444130:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.444137:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.444153:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.456134:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69885952 -> 69890048 00002000:00010000:3.0:1789760753.456140:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136520 -> 136528 00010000:00010000:3.0:1789760753.456319:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.456337:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.456347:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.456356:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.456363:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.456373:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.456795:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.456806:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eda93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69885952->69890047] (req 69885952->69890047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.465343:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760753.503551:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760753.555638:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760753.579632:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760753.610776:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69910528 -> 69914624 00002000:00010000:3.0:1789760753.610792:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136568 -> 136576 00010000:00010000:3.0:1789760753.610809:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083edabd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69910528->69914623] (req 69910528->69914623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.610823:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083edabd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69910528->69914623] (req 69910528->69914623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.610832:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083edabd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69910528->69914623] (req 69910528->69914623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.610840:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.610846:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083edabd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69910528->69914623] (req 69910528->69914623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.610856:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083edabd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69910528->69914623] (req 69910528->69914623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.610948:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083edabd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69910528->69914623] (req 69910528->69914623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.610958:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083edabd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69910528->69914623] (req 69910528->69914623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.611181:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503569792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9653us (11410us total) trans 4294984375 rc 0/0 00000100:00100000:3.0:1789760753.611194:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17079 00000100:00100000:3.0:1789760753.611207:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683706452370 time 683726107217 state 0x2 ref 0 00000100:00100000:3.0:1789760753.668557:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760753.683720:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503570304 00000100:00100000:3.0:1789760753.683734:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503570304 00000100:00100000:3.0:1789760753.683817:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683774059821 time 683793643340 state 0x0 ref 1 00000100:00100000:3.0:1789760753.683836:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683793903756 time 683793643340 state 0x1 ref 1 00000100:00100000:3.0:1789760753.683872:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683793903756 time 683808478627 state 0x0 ref 1 00000100:00100000:3.0:1789760753.683887:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503570304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.683947:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69926912, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x21598 00010000:00010000:3.0:1789760753.683958:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.683977:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.683982:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.683990:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.684005:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.693093:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69926912 -> 69931008 00002000:00010000:3.0:1789760753.693102:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136600 -> 136608 00010000:00010000:3.0:1789760753.693120:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.693134:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.693142:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.693150:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.693156:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.693166:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.693191:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.693198:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edad9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69926912->69931007] (req 69926912->69931007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.693804:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503570304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9909us (10200us total) trans 4294984379 rc 0/0 00000100:00100000:3.0:1789760753.693819:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17083 00000100:00100000:3.0:1789760753.693831:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683793903756 time 683808478627 state 0x2 ref 0 00000100:00100000:3.0:1789760753.701804:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760753.723200:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683826972250 time 683847795040 state 0x0 ref 1 00000100:00100000:3.0:1789760753.723230:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503570560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.724556:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69935104, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x215a8 00010000:00010000:3.0:1789760753.724576:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.724599:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.724605:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.724612:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.724629:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.732242:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69935104 -> 69939200 00002000:00010000:3.0:1789760753.732247:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136616 -> 136624 00010000:00010000:3.0:1789760753.732260:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732273:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732282:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732290:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.732294:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732303:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732324:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732337:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edae7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69935104->69939199] (req 69935104->69939199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.732579:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503570560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9354us (9935us total) trans 4294984381 rc 0/0 00000100:00100000:3.0:1789760753.732590:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17085 00000100:00100000:3.0:1789760753.732599:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683826972250 time 683847795040 state 0x2 ref 0 00000100:00100000:3.0:1789760753.773915:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760753.796590:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503571072 00000100:00100000:3.0:1789760753.796638:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503571072 00000100:00100000:3.0:1789760753.796717:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683880840249 time 683898823726 state 0x0 ref 1 00000100:00100000:3.0:1789760753.796740:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683899084142 time 683898823726 state 0x1 ref 1 00000100:00100000:3.0:1789760753.796789:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683899084142 time 683921395369 state 0x0 ref 1 00000100:00100000:3.0:1789760753.796804:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503571072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.796866:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69951488, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x215c8 00010000:00010000:3.0:1789760753.796877:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edb03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.796897:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=683s v=(0 0) 00010000:00010000:3.0:1789760753.796903:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.796911:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edb03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69951488->69955583] (req 69951488->69955583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.796930:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edb03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69951488->69955583] (req 69951488->69955583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.843895:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760753.871376:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984388 is committed 00000100:00100000:3.0:1789760753.881636:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760753.889885:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69967872 -> 69971968 00002000:00010000:3.0:1789760753.889890:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136680 -> 136688 00010000:00010000:3.0:1789760753.889904:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083edb1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69967872->69971967] (req 69967872->69971967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889916:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083edb1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69967872->69971967] (req 69967872->69971967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889926:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083edb1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69967872->69971967] (req 69967872->69971967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889934:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.889939:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083edb1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69967872->69971967] (req 69967872->69971967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889949:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083edb1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69967872->69971967] (req 69967872->69971967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889971:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083edb1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69967872->69971967] (req 69967872->69971967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.889982:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083edb1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69967872->69971967] (req 69967872->69971967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.890285:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503571584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8228us (8652us total) trans 4294984389 rc 0/0 00000100:00100000:3.0:1789760753.890297:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17093 00000100:00100000:3.0:1789760753.890307:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 683985312409 time 684006645502 state 0x2 ref 0 00000100:00100000:3.0:1789760753.893321:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760753.903802:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 69971968 -> 69976064 00002000:00010000:3.0:1789760753.903813:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136688 -> 136696 00010000:00010000:3.0:1789760753.903826:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083edb26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69971968->69976063] (req 69971968->69976063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.903970:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083edb26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69971968->69976063] (req 69971968->69976063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.903980:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083edb26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69971968->69976063] (req 69971968->69976063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.903988:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760753.903993:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083edb26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69971968->69976063] (req 69971968->69976063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.904003:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083edb26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69971968->69976063] (req 69971968->69976063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.904025:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083edb26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69971968->69976063] (req 69971968->69976063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.904034:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083edb26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69971968->69976063] (req 69971968->69976063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.904175:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503571712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10612us (10857us total) trans 4294984390 rc 0/0 00000100:00100000:3.0:1789760753.904186:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17094 00000100:00100000:3.0:1789760753.904196:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684006905918 time 684018153811 state 0x2 ref 0 00000100:00100000:3.0:1789760753.944064:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760753.950471:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984393 is committed 00000100:00100000:3.0:1789760753.961046:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503572224 00000100:00100000:3.0:1789760753.961059:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503572224 00000100:00100000:3.0:1789760753.961148:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684057206239 time 684068931950 state 0x0 ref 1 00000100:00100000:3.0:1789760753.961167:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684069192366 time 684068931950 state 0x1 ref 1 00000100:00100000:3.0:1789760753.961205:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684069192366 time 684085810614 state 0x0 ref 1 00000100:00100000:3.0:1789760753.961218:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503572224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760753.961286:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 69988352, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x21610 00010000:00010000:3.0:1789760753.961298:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edb42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.961326:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760753.961332:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.961340:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edb42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69988352->69992447] (req 69988352->69992447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.961357:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edb42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [69988352->69992447] (req 69988352->69992447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.967222:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984394 is committed 00000100:00100000:3.0:1789760753.980337:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760753.989929:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984395 is committed 00000100:00100000:3.0:1789760753.994178:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.000754:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984396 is committed 00000100:00100000:3.0:1789760754.012599:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503572608 00000100:00100000:3.0:1789760754.012619:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503572608 00000100:00100000:3.0:1789760754.012810:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684105457119 time 684119175061 state 0x0 ref 1 00000100:00100000:3.0:1789760754.012829:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684119435477 time 684119175061 state 0x1 ref 1 00000100:00100000:3.0:1789760754.012878:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684119435477 time 684137484319 state 0x0 ref 1 00000100:00100000:3.0:1789760754.012896:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503572608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.012962:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70000640, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x21628 00010000:00010000:3.0:1789760754.012974:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.013000:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760754.013006:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.013013:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.013030:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.021583:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70000640 -> 70004736 00002000:00010000:3.0:1789760754.021589:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136744 -> 136752 00010000:00010000:3.0:1789760754.021618:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.021633:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.021643:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.021651:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.021656:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.021666:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.021697:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.021705:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edb57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70000640->70004735] (req 70000640->70004735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.021862:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503572608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8964us (10632us total) trans 4294984397 rc 0/0 00000100:00100000:3.0:1789760754.021953:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17101 00000100:00100000:3.0:1789760754.021966:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684119435477 time 684137484319 state 0x2 ref 0 00000100:00100000:3.0:1789760754.026808:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.068107:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.077488:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984401 is committed 00000100:00100000:3.0:1789760754.083000:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.088936:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984402 is committed 00000100:00100000:3.0:1789760754.117150:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.125634:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984404 is committed 00000001:00080000:3.0:1789760754.167030:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984406 is committed 00000100:00100000:3.0:1789760754.182199:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00010000:00010000:3.0:1789760754.195241:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760754.229636:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70049792 -> 70053888 00002000:00010000:3.0:1789760754.229647:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136840 -> 136848 00010000:00010000:3.0:1789760754.229665:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edbab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70049792->70053887] (req 70049792->70053887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.229685:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edbab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70049792->70053887] (req 70049792->70053887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.229694:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edbab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70049792->70053887] (req 70049792->70053887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.229702:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.229707:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edbab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70049792->70053887] (req 70049792->70053887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.229718:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edbab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70049792->70053887] (req 70049792->70053887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.229749:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edbab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70049792->70053887] (req 70049792->70053887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.229756:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edbab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70049792->70053887] (req 70049792->70053887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.232842:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503574144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15654us (15870us total) trans 4294984409 rc 0/0 00000100:00100000:3.0:1789760754.232862:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17113 00000100:00100000:3.0:1789760754.232876:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684326293171 time 684341765234 state 0x2 ref 0 00000100:00100000:3.0:1789760754.238727:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.261017:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.296299:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.313687:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.342743:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.357209:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.363646:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984417 is committed 00000001:00080000:3.0:1789760754.388150:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984418 is committed 00000100:00100000:3.0:1789760754.396339:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.406026:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984419 is committed 00000100:00100000:3.0:1789760754.424670:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.435286:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984420 is committed 00000100:00100000:3.0:1789760754.441308:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503575680 00000100:00100000:3.0:1789760754.441322:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503575680 00000100:00100000:3.0:1789760754.441671:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684522103887 time 684553794585 state 0x0 ref 1 00000100:00100000:3.0:1789760754.441693:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684554055001 time 684553794585 state 0x1 ref 1 00000100:00100000:3.0:1789760754.441758:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684554055001 time 684566364382 state 0x0 ref 1 00000100:00100000:3.0:1789760754.441782:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503575680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.441845:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70098944, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x216e8 00010000:00010000:3.0:1789760754.441856:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.441875:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760754.441880:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.441888:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.441911:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.456051:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70098944 -> 70103040 00002000:00010000:3.0:1789760754.456064:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 136936 -> 136944 00010000:00010000:3.0:1789760754.456081:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.456095:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.456104:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.456112:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.456127:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.456138:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.456162:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.456170:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edbff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70098944->70103039] (req 70098944->70103039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.456304:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503575680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14526us (15201us total) trans 4294984421 rc 0/0 00000100:00100000:3.0:1789760754.456316:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17125 00000100:00100000:3.0:1789760754.456329:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684554055001 time 684566364382 state 0x2 ref 0 00000100:00100000:3.0:1789760754.461608:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.482180:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984423 is committed 00000100:00100000:3.0:1789760754.505573:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.529071:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.567474:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.579497:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.613717:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.628362:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.629199:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684715739278 time 684738984263 state 0x0 ref 1 00000100:00100000:3.0:1789760754.629220:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684739244679 time 684738984263 state 0x1 ref 1 00000100:00100000:3.0:1789760754.629291:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684739244679 time 684753896970 state 0x0 ref 1 00000100:00100000:3.0:1789760754.629306:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503577088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.629382:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70144000, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x21740 00010000:00010000:3.0:1789760754.629421:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.629442:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760754.629448:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.629455:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.629473:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.640482:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70144000 -> 70148096 00002000:00010000:3.0:1789760754.640490:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137024 -> 137032 00010000:00010000:3.0:1789760754.640509:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.640523:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.640533:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.640540:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.640546:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.640557:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.640582:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.640596:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70144000->70148095] (req 70144000->70148095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.640731:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503577088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11420us (12377us total) trans 4294984432 rc 0/0 00000100:00100000:3.0:1789760754.640744:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17136 00000100:00100000:3.0:1789760754.640758:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684739244679 time 684753896970 state 0x2 ref 0 00002000:00010000:3.0:1789760754.696376:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70152192 -> 70156288 00002000:00010000:3.0:1789760754.696411:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137040 -> 137048 00010000:00010000:3.0:1789760754.696430:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083edc5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70152192->70156287] (req 70152192->70156287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.696447:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083edc5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70152192->70156287] (req 70152192->70156287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.696457:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083edc5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70152192->70156287] (req 70152192->70156287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.696466:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.696472:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083edc5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70152192->70156287] (req 70152192->70156287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.696482:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083edc5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70152192->70156287] (req 70152192->70156287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.698228:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083edc5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70152192->70156287] (req 70152192->70156287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.698254:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083edc5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70152192->70156287] (req 70152192->70156287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.698443:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503577344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 25898us (26662us total) trans 4294984434 rc 0/0 00000100:00100000:3.0:1789760754.698456:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17138 00000100:00100000:3.0:1789760754.698469:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684773494731 time 684797081658 state 0x2 ref 0 00000100:00100000:3.0:1789760754.703366:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.741142:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503577728 00000100:00100000:3.0:1789760754.741156:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503577728 00000100:00100000:3.0:1789760754.741292:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684830196186 time 684846910071 state 0x0 ref 1 00000100:00100000:3.0:1789760754.741313:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684847170487 time 684846910071 state 0x1 ref 1 00000100:00100000:3.0:1789760754.741359:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684847170487 time 684865964889 state 0x0 ref 1 00000100:00100000:3.0:1789760754.741374:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503577728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.741495:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70164480, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x21768 00010000:00010000:3.0:1789760754.741519:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.741545:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760754.741551:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.741560:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.741577:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.753237:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70164480 -> 70168576 00002000:00010000:3.0:1789760754.753249:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137064 -> 137072 00010000:00010000:3.0:1789760754.753271:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.753286:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.753295:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.753303:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.753309:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.753319:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.753348:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.753355:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70164480->70168575] (req 70164480->70168575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.753534:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503577728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12155us (12478us total) trans 4294984437 rc 0/0 00000100:00100000:3.0:1789760754.753546:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17141 00000100:00100000:3.0:1789760754.753560:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684847170487 time 684865964889 state 0x2 ref 0 00000100:00100000:3.0:1789760754.758090:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.790038:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503578112 00000100:00100000:3.0:1789760754.790053:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503578112 00000100:00100000:3.0:1789760754.790138:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684883550678 time 684898762750 state 0x0 ref 1 00000100:00100000:3.0:1789760754.790156:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684899023166 time 684898762750 state 0x1 ref 1 00000100:00100000:3.0:1789760754.790195:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684899023166 time 684914801284 state 0x0 ref 1 00000100:00100000:3.0:1789760754.790210:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503578112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.790283:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70176768, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x21780 00010000:00010000:3.0:1789760754.790294:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.790320:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760754.790326:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.790333:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.790352:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.804865:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70176768 -> 70180864 00002000:00010000:3.0:1789760754.804879:0:15091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760753 -> 1789760754 00002000:00010000:3.0:1789760754.804882:0:15091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760753 -> 1789760754 00002000:00010000:3.0:1789760754.804885:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137088 -> 137096 00010000:00010000:3.0:1789760754.804904:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.804919:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.804930:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.805597:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.805605:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.805618:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.805647:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.805657:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edc84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70176768->70180863] (req 70176768->70180863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.806144:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503578112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15927us (16383us total) trans 4294984440 rc 0/0 00000100:00100000:3.0:1789760754.806156:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17144 00000100:00100000:3.0:1789760754.806169:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684899023166 time 684914801284 state 0x2 ref 0 00000100:00100000:3.0:1789760754.815997:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.834823:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503578368 00000100:00100000:3.0:1789760754.834838:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503578368 00000100:00100000:3.0:1789760754.834913:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684915061700 time 684941112761 state 0x0 ref 1 00000100:00100000:3.0:1789760754.834932:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684941373177 time 684941112761 state 0x1 ref 1 00000100:00100000:3.0:1789760754.834969:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684941373177 time 684959574611 state 0x0 ref 1 00000100:00100000:3.0:1789760754.834983:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503578368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.835049:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70184960, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21790 00010000:00010000:3.0:1789760754.835061:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.835090:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=684s v=(0 0) 00010000:00010000:3.0:1789760754.835096:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.835105:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.835123:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.841188:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984442 is committed 00002000:00010000:3.0:1789760754.843250:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70184960 -> 70189056 00002000:00010000:3.0:1789760754.843263:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137104 -> 137112 00010000:00010000:3.0:1789760754.843281:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.843298:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.843307:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.843315:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.843319:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.843329:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.843359:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.843367:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edc92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70184960->70189055] (req 70184960->70189055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.843618:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503578368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8629us (9096us total) trans 4294984442 rc 0/0 00000100:00100000:3.0:1789760754.843630:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17146 00000100:00100000:3.0:1789760754.843644:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684941373177 time 684959574611 state 0x2 ref 0 00000100:00100000:3.0:1789760754.861553:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760754.868921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984443 is committed 00000100:00100000:3.0:1789760754.904183:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503578752 00000100:00100000:3.0:1789760754.904198:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503578752 00000100:00100000:3.0:1789760754.904277:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 684987014153 time 685005711075 state 0x0 ref 1 00000100:00100000:3.0:1789760754.904297:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685005971491 time 685005711075 state 0x1 ref 1 00000100:00100000:3.0:1789760754.919715:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760754.935316:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503579008 00000100:00100000:3.0:1789760754.935327:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503579008 00000100:00100000:3.0:1789760754.935384:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685029352126 time 685047077549 state 0x0 ref 1 00000100:00100000:3.0:1789760754.935420:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685047337965 time 685047077549 state 0x1 ref 1 00000100:00100000:3.0:1789760754.935457:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685047337965 time 685060063230 state 0x0 ref 1 00000100:00100000:3.0:1789760754.935471:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503579008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.935524:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70205440, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x217b8 00010000:00010000:3.0:1789760754.935534:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.935549:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760754.935553:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.935559:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.935572:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.950606:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70205440 -> 70209536 00002000:00010000:3.0:1789760754.950612:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137144 -> 137152 00010000:00010000:3.0:1789760754.950625:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.950637:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.950645:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.950653:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.950658:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.950668:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.950689:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.950697:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70205440->70209535] (req 70205440->70209535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.950855:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503579008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15381us (15869us total) trans 4294984447 rc 0/0 00000100:00100000:3.0:1789760754.950866:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17151 00000100:00100000:3.0:1789760754.950877:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685047337965 time 685060063230 state 0x2 ref 0 00000100:00100000:3.0:1789760754.956113:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503579136 00000100:00100000:3.0:1789760754.956123:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503579136 00000100:00100000:3.0:1789760754.956179:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685047337965 time 685060063230 state 0x0 ref 1 00000100:00100000:3.0:1789760754.956195:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685060323646 time 685060063230 state 0x1 ref 1 00000100:00100000:3.0:1789760754.956228:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685060323646 time 685080833997 state 0x0 ref 1 00000100:00100000:3.0:1789760754.956241:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503579136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.956290:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70209536, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x217c0 00010000:00010000:3.0:1789760754.956300:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edcbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.956316:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760754.956321:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.956328:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edcbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70209536->70213631] (req 70209536->70213631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.956343:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edcbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70209536->70213631] (req 70209536->70213631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.980227:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503579392 00000100:00100000:3.0:1789760754.980238:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503579392 00000100:00100000:3.0:1789760754.980356:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685081094413 time 685095485432 state 0x0 ref 1 00000100:00100000:3.0:1789760754.980371:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685095745848 time 685095485432 state 0x1 ref 1 00000100:00100000:3.0:1789760754.980433:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685095745848 time 685105039409 state 0x0 ref 1 00000100:00100000:3.0:1789760754.980447:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503579392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.980494:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70217728, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x217d0 00010000:00010000:3.0:1789760754.980504:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.980520:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760754.980524:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.980530:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.980544:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.989274:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70217728 -> 70221824 00002000:00010000:3.0:1789760754.989280:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137168 -> 137176 00010000:00010000:3.0:1789760754.989293:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.989306:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.989314:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.989320:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760754.989325:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.989333:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.989353:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.989360:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70217728->70221823] (req 70217728->70221823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.989682:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503579392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9232us (9552us total) trans 4294984450 rc 0/0 00000100:00100000:3.0:1789760754.989692:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17154 00000100:00100000:3.0:1789760754.989702:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685095745848 time 685105039409 state 0x2 ref 0 00000100:00100000:3.0:1789760754.997029:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503579520 00000100:00100000:3.0:1789760754.997039:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503579520 00000100:00100000:3.0:1789760754.997094:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685095745848 time 685105039409 state 0x0 ref 1 00000100:00100000:3.0:1789760754.997110:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685105299825 time 685105039409 state 0x1 ref 1 00000100:00100000:3.0:1789760754.997149:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685105299825 time 685121755486 state 0x0 ref 1 00000100:00100000:3.0:1789760754.997162:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503579520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760754.997209:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70221824, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x217d8 00010000:00010000:3.0:1789760754.997219:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.997235:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760754.997240:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.997246:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.997260:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.004120:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70221824 -> 70225920 00002000:00010000:3.0:1789760755.004125:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137176 -> 137184 00010000:00010000:3.0:1789760755.004207:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.004232:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.004241:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.004248:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.004252:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.004261:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.004282:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.004290:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edcd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70221824->70225919] (req 70221824->70225919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.004474:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503579520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7309us (7542us total) trans 4294984451 rc 0/0 00000100:00100000:3.0:1789760755.004485:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17155 00000100:00100000:3.0:1789760755.004504:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685105299825 time 685121755486 state 0x2 ref 0 00000100:00100000:3.0:1789760755.044132:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503579904 00000100:00100000:3.0:1789760755.044142:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503579904 00000100:00100000:3.0:1789760755.044196:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685133167873 time 685156126500 state 0x0 ref 1 00000100:00100000:3.0:1789760755.044211:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685156386916 time 685156126500 state 0x1 ref 1 00000100:00100000:3.0:1789760755.044241:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685156386916 time 685168847335 state 0x0 ref 1 00000100:00100000:3.0:1789760755.044253:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503579904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.044299:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70234112, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x217f0 00010000:00010000:3.0:1789760755.044308:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083edce6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.044324:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.044328:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.044334:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083edce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70234112->70238207] (req 70234112->70238207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.044348:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083edce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70234112->70238207] (req 70234112->70238207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.054940:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984454 is committed 00000100:00100000:3.0:1789760755.060887:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760755.073026:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760755.095119:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760755.151324:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503580928 00000100:00100000:3.0:1789760755.151335:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503580928 00000100:00100000:3.0:1789760755.151420:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685246392078 time 685266707016 state 0x0 ref 1 00000100:00100000:3.0:1789760755.151436:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685266967432 time 685266707016 state 0x1 ref 1 00000100:00100000:3.0:1789760755.151459:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685266967432 time 685276066215 state 0x0 ref 1 00000100:00100000:3.0:1789760755.151471:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff518050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503580928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.151527:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70266880, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21830 00010000:00010000:3.0:1789760755.151537:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.151553:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.151558:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.151564:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.151578:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.155774:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70266880 -> 70270976 00002000:00010000:3.0:1789760755.155780:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137264 -> 137272 00010000:00010000:3.0:1789760755.155792:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.155818:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.155828:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.155835:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.155840:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.155850:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.155870:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.155878:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70266880->70270975] (req 70266880->70270975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.155971:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff518050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503580928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4498us (4736us total) trans 4294984462 rc 0/0 00000100:00100000:3.0:1789760755.155982:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17166 00000100:00100000:3.0:1789760755.155992:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685266967432 time 685276066215 state 0x2 ref 0 00000100:00100000:3.0:1789760755.159891:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503581056 00000100:00100000:3.0:1789760755.159900:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503581056 00000100:00100000:3.0:1789760755.159957:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685266967432 time 685276066215 state 0x0 ref 1 00000100:00100000:3.0:1789760755.159973:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685276326631 time 685276066215 state 0x1 ref 1 00000100:00100000:3.0:1789760755.160047:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685276326631 time 685284653256 state 0x0 ref 1 00000100:00100000:3.0:1789760755.160062:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503581056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.160106:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70270976, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21838 00010000:00010000:3.0:1789760755.160115:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.160131:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.160135:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.160141:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.160154:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.167864:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70270976 -> 70275072 00002000:00010000:3.0:1789760755.167874:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137272 -> 137280 00010000:00010000:3.0:1789760755.167888:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.167901:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.167915:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.167923:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.167928:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.167937:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.167961:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.167969:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70270976->70275071] (req 70270976->70275071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.168111:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503581056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8047us (8373us total) trans 4294984463 rc 0/0 00000100:00100000:3.0:1789760755.168122:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17167 00000100:00100000:3.0:1789760755.168132:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685276326631 time 685284653256 state 0x2 ref 0 00000100:00100000:3.0:1789760755.199372:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503581440 00000100:00100000:3.0:1789760755.199383:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503581440 00000100:00100000:3.0:1789760755.199491:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685296228112 time 685311704822 state 0x0 ref 1 00000100:00100000:3.0:1789760755.199507:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685311965238 time 685311704822 state 0x1 ref 1 00000100:00100000:3.0:1789760755.199544:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685311965238 time 685324150273 state 0x0 ref 1 00000100:00100000:3.0:1789760755.199558:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503581440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.199606:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70283264, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21850 00010000:00010000:3.0:1789760755.199615:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.199631:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.199636:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.199643:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70283264->70287359] (req 70283264->70287359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.199662:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083edd3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70283264->70287359] (req 70283264->70287359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.217326:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503581568 00000100:00100000:3.0:1789760755.217339:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503581568 00000100:00100000:3.0:1789760755.217442:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685311965238 time 685324150273 state 0x0 ref 1 00000100:00100000:3.0:1789760755.217460:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685324410689 time 685324150273 state 0x1 ref 1 00000100:00100000:3.0:1789760755.217499:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685324410689 time 685342105427 state 0x0 ref 1 00000100:00100000:3.0:1789760755.217512:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503581568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.217564:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70287360, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21858 00010000:00010000:3.0:1789760755.217574:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.217589:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.217730:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.217937:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.217954:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.218795:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760755.223990:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70287360 -> 70291456 00002000:00010000:3.0:1789760755.223996:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137304 -> 137312 00010000:00010000:3.0:1789760755.224009:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.224021:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.224031:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.224038:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.224043:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.224052:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.224073:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.224081:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70287360->70291455] (req 70287360->70291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.224189:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503581568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6674us (7209us total) trans 4294984467 rc 0/0 00000100:00100000:3.0:1789760755.224199:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17171 00000100:00100000:3.0:1789760755.224208:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685324410689 time 685342105427 state 0x2 ref 0 00000100:00100000:3.0:1789760755.227987:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760755.240375:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503581824 00000100:00100000:3.0:1789760755.240384:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503581824 00000100:00100000:3.0:1789760755.240468:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685342365843 time 685352857861 state 0x0 ref 1 00000100:00100000:3.0:1789760755.240483:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685353118277 time 685352857861 state 0x1 ref 1 00000100:00100000:3.0:1789760755.240515:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685353118277 time 685365121582 state 0x0 ref 1 00000100:00100000:3.0:1789760755.240528:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503581824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.240574:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70295552, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21868 00010000:00010000:3.0:1789760755.240583:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.240604:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.240609:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.240616:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.240629:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.244982:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70295552 -> 70299648 00002000:00010000:3.0:1789760755.244987:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137320 -> 137328 00010000:00010000:3.0:1789760755.244999:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.245013:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.245023:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.245030:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.245036:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.245045:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.245068:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.245077:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70295552->70299647] (req 70295552->70299647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.245360:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503581824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4829us (5064us total) trans 4294984469 rc 0/0 00000100:00100000:3.0:1789760755.245371:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17173 00000100:00100000:3.0:1789760755.245381:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685353118277 time 685365121582 state 0x2 ref 0 00000100:00100000:3.0:1789760755.272079:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503582080 00000100:00100000:3.0:1789760755.272090:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503582080 00000100:00100000:3.0:1789760755.272144:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685365381998 time 685373413041 state 0x0 ref 1 00000100:00100000:3.0:1789760755.272160:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685373673457 time 685373413041 state 0x1 ref 1 00000100:00100000:3.0:1789760755.272193:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685373673457 time 685396799760 state 0x0 ref 1 00000100:00100000:3.0:1789760755.272207:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503582080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.272257:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70303744, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21878 00010000:00010000:3.0:1789760755.272267:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272283:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.272289:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.272295:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272308:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.277999:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70303744 -> 70307840 00002000:00010000:3.0:1789760755.278005:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137336 -> 137344 00010000:00010000:3.0:1789760755.278019:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278034:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278043:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278050:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.278056:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278066:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278087:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.278101:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70303744->70307839] (req 70303744->70307839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.278210:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503582080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6000us (6293us total) trans 4294984471 rc 0/0 00000100:00100000:3.0:1789760755.278222:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17175 00000100:00100000:3.0:1789760755.278232:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685373673457 time 685396799760 state 0x2 ref 0 00000100:00100000:3.0:1789760755.316554:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503582592 00000100:00100000:3.0:1789760755.316567:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503582592 00000100:00100000:3.0:1789760755.316624:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685418817298 time 685430997582 state 0x0 ref 1 00000100:00100000:3.0:1789760755.316640:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685431257998 time 685430997582 state 0x1 ref 1 00000100:00100000:3.0:1789760755.316671:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685431257998 time 685441278159 state 0x0 ref 1 00000100:00100000:3.0:1789760755.316683:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503582592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.316728:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70320128, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21898 00010000:00010000:3.0:1789760755.316737:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.316752:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.316756:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.316762:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.316792:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.322884:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70320128 -> 70324224 00002000:00010000:3.0:1789760755.322890:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137368 -> 137376 00010000:00010000:3.0:1789760755.322904:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.322918:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.322927:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.322934:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.322939:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.322948:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.322969:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.322978:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70320128->70324223] (req 70320128->70324223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.323122:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503582592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6436us (6801us total) trans 4294984475 rc 0/0 00000100:00100000:3.0:1789760755.323134:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17179 00000100:00100000:3.0:1789760755.323144:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685431257998 time 685441278159 state 0x2 ref 0 00000100:00100000:3.0:1789760755.358833:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503583104 00000100:00100000:3.0:1789760755.358843:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503583104 00000100:00100000:3.0:1789760755.358895:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685465115347 time 685475360111 state 0x0 ref 1 00000100:00100000:3.0:1789760755.358910:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685475620527 time 685475360111 state 0x1 ref 1 00000100:00100000:3.0:1789760755.358943:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685475620527 time 685483549052 state 0x0 ref 1 00000100:00100000:3.0:1789760755.358965:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503583104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.359015:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70336512, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x218b8 00010000:00010000:3.0:1789760755.359024:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.359040:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.359044:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.359051:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.359064:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.366573:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70336512 -> 70340608 00002000:00010000:3.0:1789760755.366578:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137400 -> 137408 00010000:00010000:3.0:1789760755.366591:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.366615:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.366624:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.366632:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.366637:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.366647:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.366669:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.366677:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edd95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70336512->70340607] (req 70336512->70340607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.366802:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503583104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7843us (8129us total) trans 4294984479 rc 0/0 00000100:00100000:3.0:1789760755.366813:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17183 00000100:00100000:3.0:1789760755.366823:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685475620527 time 685483549052 state 0x2 ref 0 00000100:00100000:3.0:1789760755.403360:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503583616 00000100:00100000:3.0:1789760755.403369:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503583616 00000100:00100000:3.0:1789760755.403456:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685510218314 time 685519287208 state 0x0 ref 1 00000100:00100000:3.0:1789760755.403472:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685519547624 time 685519287208 state 0x1 ref 1 00000100:00100000:3.0:1789760755.403503:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685519547624 time 685528109829 state 0x0 ref 1 00000100:00100000:3.0:1789760755.403516:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503583616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.403562:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70352896, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x218d8 00010000:00010000:3.0:1789760755.403572:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.403587:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.403592:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.403598:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.403612:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.407665:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70352896 -> 70356992 00002000:00010000:3.0:1789760755.407671:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137432 -> 137440 00010000:00010000:3.0:1789760755.407686:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.407699:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.407708:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.407716:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.407721:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.407729:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.407748:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.407756:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70352896->70356991] (req 70352896->70356991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.407881:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503583616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4362us (4695us total) trans 4294984483 rc 0/0 00000100:00100000:3.0:1789760755.407892:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17187 00000100:00100000:3.0:1789760755.407903:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685519547624 time 685528109829 state 0x2 ref 0 00000100:00100000:3.0:1789760755.440543:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503584128 00000100:00100000:3.0:1789760755.440554:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503584128 00000100:00100000:3.0:1789760755.440686:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685544934175 time 685553400480 state 0x0 ref 1 00000100:00100000:3.0:1789760755.440702:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685553660896 time 685553400480 state 0x1 ref 1 00000100:00100000:3.0:1789760755.440790:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685553660896 time 685565396026 state 0x0 ref 1 00000100:00100000:3.0:1789760755.440800:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503584128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.440845:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70369280, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x218f8 00010000:00010000:3.0:1789760755.440855:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.440868:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.440871:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.440876:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.440886:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.446125:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70369280 -> 70373376 00002000:00010000:3.0:1789760755.446131:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137464 -> 137472 00010000:00010000:3.0:1789760755.446143:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.446162:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.446172:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.446179:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.446185:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.446194:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.446279:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.446288:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eddcd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70369280->70373375] (req 70369280->70373375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.446556:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503584128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5751us (6217us total) trans 4294984487 rc 0/0 00000100:00100000:3.0:1789760755.446567:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17191 00000100:00100000:3.0:1789760755.446576:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685553660896 time 685565396026 state 0x2 ref 0 00000001:00080000:3.0:1789760755.478697:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984490 is committed 00000100:00100000:3.0:1789760755.488304:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503584640 00000100:00100000:3.0:1789760755.488314:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503584640 00000100:00100000:3.0:1789760755.488366:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685585993118 time 685597863822 state 0x0 ref 1 00000100:00100000:3.0:1789760755.488382:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685598124238 time 685597863822 state 0x1 ref 1 00000100:00100000:3.0:1789760755.488990:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685598124238 time 685613593418 state 0x0 ref 1 00000100:00100000:3.0:1789760755.489007:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503584640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.489067:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70385664, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21918 00010000:00010000:3.0:1789760755.489077:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.489092:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.489097:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.489104:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.489118:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.494047:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70385664 -> 70389760 00002000:00010000:3.0:1789760755.494053:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137496 -> 137504 00010000:00010000:3.0:1789760755.494065:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.494078:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.494087:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.494095:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.494099:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.494110:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.494129:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.494137:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083edde9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70385664->70389759] (req 70385664->70389759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.494244:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503584640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5235us (6034us total) trans 4294984491 rc 0/0 00000100:00100000:3.0:1789760755.494255:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17195 00000100:00100000:3.0:1789760755.494264:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685598124238 time 685613593418 state 0x2 ref 0 00000001:00080000:3.0:1789760755.531348:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984494 is committed 00000100:00100000:3.0:1789760755.535695:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503585152 00000100:00100000:3.0:1789760755.535706:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503585152 00000100:00100000:3.0:1789760755.535760:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685640908113 time 685650898124 state 0x0 ref 1 00000100:00100000:3.0:1789760755.535790:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685651158540 time 685650898124 state 0x1 ref 1 00000100:00100000:3.0:1789760755.535954:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685651158540 time 685660559575 state 0x0 ref 1 00000100:00100000:3.0:1789760755.535968:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503585152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.536024:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70402048, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21938 00010000:00010000:3.0:1789760755.536034:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.536049:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.536053:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.536059:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.536073:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.545424:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70402048 -> 70406144 00002000:00010000:3.0:1789760755.545430:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137528 -> 137536 00010000:00010000:3.0:1789760755.545444:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.545457:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.545467:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.545476:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.545481:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.545556:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.545577:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.545584:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70402048->70406143] (req 70402048->70406143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.545698:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503585152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9727us (10102us total) trans 4294984495 rc 0/0 00000100:00100000:3.0:1789760755.545709:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17199 00000100:00100000:3.0:1789760755.545720:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685651158540 time 685660559575 state 0x2 ref 0 00002000:00010000:3.0:1789760755.568702:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70410240 -> 70414336 00002000:00010000:3.0:1789760755.568708:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137544 -> 137552 00010000:00010000:3.0:1789760755.568723:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ede13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70410240->70414335] (req 70410240->70414335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.568735:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ede13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70410240->70414335] (req 70410240->70414335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.568750:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ede13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70410240->70414335] (req 70410240->70414335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.568757:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.568975:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ede13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70410240->70414335] (req 70410240->70414335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.568987:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ede13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70410240->70414335] (req 70410240->70414335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.569005:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ede13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70410240->70414335] (req 70410240->70414335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.569013:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ede13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70410240->70414335] (req 70410240->70414335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.569149:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503585408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6430us (6679us total) trans 4294984497 rc 0/0 00000100:00100000:3.0:1789760755.569160:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17201 00000100:00100000:3.0:1789760755.569169:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685677039813 time 685687310248 state 0x2 ref 0 00000001:00080000:3.0:1789760755.581651:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984498 is committed 00000100:00100000:3.0:1789760755.587105:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503585664 00000100:00100000:3.0:1789760755.587115:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503585664 00000100:00100000:3.0:1789760755.587169:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685687570664 time 685700396675 state 0x0 ref 1 00000100:00100000:3.0:1789760755.587187:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685700657091 time 685700396675 state 0x1 ref 1 00000100:00100000:3.0:1789760755.587220:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685700657091 time 685711826924 state 0x0 ref 1 00000100:00100000:3.0:1789760755.587232:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503585664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.587276:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70418432, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x21958 00010000:00010000:3.0:1789760755.587284:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.587300:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.587304:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.587310:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70418432->70422527] (req 70418432->70422527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.587512:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ede21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70418432->70422527] (req 70418432->70422527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.591960:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984499 is committed 00000001:00080000:3.0:1789760755.613168:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984500 is committed 00000001:00080000:3.0:1789760755.621104:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984501 is committed 00000100:00100000:3.0:1789760755.713925:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503586816 00000100:00100000:3.0:1789760755.713936:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503586816 00000100:00100000:3.0:1789760755.713988:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685804940791 time 685821159345 state 0x0 ref 1 00000100:00100000:3.0:1789760755.714003:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685821419761 time 685821159345 state 0x1 ref 1 00000100:00100000:3.0:1789760755.714037:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685821419761 time 685838643812 state 0x0 ref 1 00000100:00100000:3.0:1789760755.714050:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503586816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.714095:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70455296, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x219a0 00010000:00010000:3.0:1789760755.714104:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.714120:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.714125:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.714132:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.714146:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.721649:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70455296 -> 70459392 00002000:00010000:3.0:1789760755.721655:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137632 -> 137640 00010000:00010000:3.0:1789760755.721679:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.721943:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.721955:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.721962:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.721968:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.721978:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.721999:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.722007:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70455296->70459391] (req 70455296->70459391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.722093:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503586816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8040us (8366us total) trans 4294984508 rc 0/0 00000100:00100000:3.0:1789760755.722103:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17212 00000100:00100000:3.0:1789760755.722113:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685821419761 time 685838643812 state 0x2 ref 0 00000100:00100000:3.0:1789760755.785770:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503587328 00000100:00100000:3.0:1789760755.785783:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503587328 00000100:00100000:3.0:1789760755.785912:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685872497752 time 685892615645 state 0x0 ref 1 00000100:00100000:3.0:1789760755.785928:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685892876061 time 685892615645 state 0x1 ref 1 00000100:00100000:3.0:1789760755.786028:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685892876061 time 685910633284 state 0x0 ref 1 00000100:00100000:3.0:1789760755.786042:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503587328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.786094:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70471680, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x219c0 00010000:00010000:3.0:1789760755.786104:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.786121:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.786126:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.786132:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.786145:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.794559:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70471680 -> 70475776 00002000:00010000:3.0:1789760755.794565:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760754 -> 1789760755 00002000:00010000:3.0:1789760755.794568:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760754 -> 1789760755 00002000:00010000:3.0:1789760755.794570:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137664 -> 137672 00010000:00010000:3.0:1789760755.794585:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.794599:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.794608:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.794616:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.794621:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.794636:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.794656:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.794664:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70471680->70475775] (req 70471680->70475775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.794830:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503587328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8786us (9687us total) trans 4294984512 rc 0/0 00000100:00100000:3.0:1789760755.794841:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17216 00000100:00100000:3.0:1789760755.794851:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685892876061 time 685910633284 state 0x2 ref 0 00000100:00100000:3.0:1789760755.851957:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503587840 00000100:00100000:3.0:1789760755.851968:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503587840 00000100:00100000:3.0:1789760755.852026:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685943073693 time 685960509053 state 0x0 ref 1 00000100:00100000:3.0:1789760755.852041:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685960769469 time 685960509053 state 0x1 ref 1 00000100:00100000:3.0:1789760755.852076:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685960769469 time 685976682160 state 0x0 ref 1 00000100:00100000:3.0:1789760755.852090:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503587840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.852227:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70488064, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x219e0 00010000:00010000:3.0:1789760755.852239:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.852256:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=685s v=(0 0) 00010000:00010000:3.0:1789760755.852260:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.852267:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.852282:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.861298:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70488064 -> 70492160 00002000:00010000:3.0:1789760755.861303:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137696 -> 137704 00010000:00010000:3.0:1789760755.861316:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.861455:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.861465:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.861473:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.861477:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.861533:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.861558:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.861566:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ede98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70488064->70492159] (req 70488064->70492159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.883874:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70496256 -> 70500352 00002000:00010000:3.0:1789760755.883879:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137712 -> 137720 00010000:00010000:3.0:1789760755.883891:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083edea6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70496256->70500351] (req 70496256->70500351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.883904:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083edea6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70496256->70500351] (req 70496256->70500351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.883914:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083edea6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70496256->70500351] (req 70496256->70500351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.883922:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.883927:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083edea6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70496256->70500351] (req 70496256->70500351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.883937:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083edea6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70496256->70500351] (req 70496256->70500351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.883958:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083edea6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70496256->70500351] (req 70496256->70500351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.883967:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083edea6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70496256->70500351] (req 70496256->70500351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.884063:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503588096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5458us (5720us total) trans 4294984518 rc 0/0 00000100:00100000:3.0:1789760755.884075:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17222 00000100:00100000:3.0:1789760755.884086:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 685991424763 time 686003191980 state 0x2 ref 0 00000100:00100000:3.0:1789760755.922940:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523598976 00000100:00100000:3.0:1789760755.922950:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523598976 00000100:00100000:3.0:1789760755.923007:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:3.0:1789760755.923016:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523598976:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760755.923050:0:6517:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:3.0:1789760755.923109:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523598976:12345-0@lo:41:osp-pre-1-0.0 Request processed in 94us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760755.923115:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:3.0:1789760755.926338:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503588608 00000100:00100000:3.0:1789760755.926347:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503588608 00000100:00100000:3.0:1789760755.926437:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686026428639 time 686037672975 state 0x0 ref 1 00000100:00100000:3.0:1789760755.926453:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686037933391 time 686037672975 state 0x1 ref 1 00000100:00100000:3.0:1789760755.926474:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686037933391 time 686051080977 state 0x0 ref 1 00000100:00100000:3.0:1789760755.926486:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503588608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.926535:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70512640, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a10 00010000:00010000:3.0:1789760755.926544:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.926560:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760755.926565:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.926572:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.926587:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.931910:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70512640 -> 70516736 00002000:00010000:3.0:1789760755.931916:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137744 -> 137752 00010000:00010000:3.0:1789760755.931927:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.931938:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.931946:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.932013:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.932018:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.932031:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.932534:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.932542:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edec2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70512640->70516735] (req 70512640->70516735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.932670:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0777100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503588608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6183us (6489us total) trans 4294984522 rc 0/0 00000100:00100000:3.0:1789760755.932680:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17226 00000100:00100000:3.0:1789760755.932689:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686037933391 time 686051080977 state 0x2 ref 0 00002000:00010000:3.0:1789760755.971040:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70524928 -> 70529024 00002000:00010000:3.0:1789760755.971045:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137768 -> 137776 00010000:00010000:3.0:1789760755.971066:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eded7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70524928->70529023] (req 70524928->70529023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.971080:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eded7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70524928->70529023] (req 70524928->70529023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.971088:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eded7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70524928->70529023] (req 70524928->70529023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.971101:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760755.971106:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eded7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70524928->70529023] (req 70524928->70529023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.971115:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eded7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70524928->70529023] (req 70524928->70529023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.971135:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eded7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70524928->70529023] (req 70524928->70529023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.971143:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eded7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70524928->70529023] (req 70524928->70529023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.971467:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503588992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7015us (7477us total) trans 4294984525 rc 0/0 00000100:00100000:3.0:1789760755.971479:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17229 00000100:00100000:3.0:1789760755.971490:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686072452640 time 686089038799 state 0x2 ref 0 00000100:00100000:3.0:1789760755.984650:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503589120 00000100:00100000:3.0:1789760755.984660:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503589120 00000100:00100000:3.0:1789760755.984941:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686072452640 time 686089038799 state 0x0 ref 1 00000100:00100000:3.0:1789760755.984958:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686089299215 time 686089038799 state 0x1 ref 1 00000100:00100000:3.0:1789760755.984991:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686089299215 time 686109596880 state 0x0 ref 1 00000100:00100000:3.0:1789760755.985007:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503589120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760755.985056:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70529024, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a30 00010000:00010000:3.0:1789760755.985065:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edede lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.985470:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760755.985476:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.985484:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70529024->70533119] (req 70529024->70533119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.985500:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083edede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70529024->70533119] (req 70529024->70533119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.990381:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984526 is committed 00000001:00080000:3.0:1789760755.999373:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984527 is committed 00000001:00080000:3.0:1789760756.008433:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984528 is committed 00000100:00100000:3.0:1789760756.017010:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503589504 00000100:00100000:3.0:1789760756.017023:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503589504 00000100:00100000:3.0:1789760756.017090:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686120071780 time 686128560390 state 0x0 ref 1 00000100:00100000:3.0:1789760756.017106:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686128820806 time 686128560390 state 0x1 ref 1 00000100:00100000:3.0:1789760756.017141:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686128820806 time 686141747048 state 0x0 ref 1 00000100:00100000:3.0:1789760756.017153:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503589504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.017199:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70541312, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a48 00010000:00010000:3.0:1789760756.017209:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edef3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.017224:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.017229:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.017236:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70541312->70545407] (req 70541312->70545407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.017250:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70541312->70545407] (req 70541312->70545407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.036370:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503589632 00000100:00100000:3.0:1789760756.036385:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503589632 00000100:00100000:3.0:1789760756.036644:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686128820806 time 686141747048 state 0x0 ref 1 00000100:00100000:3.0:1789760756.036660:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686142007464 time 686141747048 state 0x1 ref 1 00000100:00100000:3.0:1789760756.036799:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686142007464 time 686161404178 state 0x0 ref 1 00000100:00100000:3.0:1789760756.036816:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503589632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.036876:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70545408, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a50 00010000:00010000:3.0:1789760756.036887:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.036904:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.036909:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.036916:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.036932:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.042180:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70545408 -> 70549504 00002000:00010000:3.0:1789760756.042186:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137808 -> 137816 00010000:00010000:3.0:1789760756.042267:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.042280:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.042289:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.042296:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.042301:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.042310:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.042330:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.042338:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edefa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70545408->70549503] (req 70545408->70549503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.042667:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503589632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5851us (8879us total) trans 4294984530 rc 0/0 00000100:00100000:3.0:1789760756.042687:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17234 00000100:00100000:3.0:1789760756.042697:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686142007464 time 686161404178 state 0x2 ref 0 00002000:00010000:3.0:1789760756.055603:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70549504 -> 70553600 00002000:00010000:3.0:1789760756.055608:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137816 -> 137824 00010000:00010000:3.0:1789760756.055620:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edf01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70549504->70553599] (req 70549504->70553599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.055632:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edf01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70549504->70553599] (req 70549504->70553599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.055641:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edf01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70549504->70553599] (req 70549504->70553599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.055648:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.055653:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edf01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70549504->70553599] (req 70549504->70553599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.055663:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edf01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70549504->70553599] (req 70549504->70553599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.055683:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edf01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70549504->70553599] (req 70549504->70553599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.055697:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083edf01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70549504->70553599] (req 70549504->70553599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.055857:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503589760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9484us (9705us total) trans 4294984531 rc 0/0 00000100:00100000:3.0:1789760756.055870:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17235 00000100:00100000:3.0:1789760756.055880:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686161664594 time 686170901713 state 0x2 ref 0 00000100:00100000:3.0:1789760756.083091:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503590144 00000100:00100000:3.0:1789760756.083102:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503590144 00000100:00100000:3.0:1789760756.083229:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686186716198 time 686196282879 state 0x0 ref 1 00000100:00100000:3.0:1789760756.083243:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686196543295 time 686196282879 state 0x1 ref 1 00000100:00100000:3.0:1789760756.083312:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686196543295 time 686207918443 state 0x0 ref 1 00000100:00100000:3.0:1789760756.083325:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503590144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.083374:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70561792, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a70 00010000:00010000:3.0:1789760756.083383:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.083422:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.083427:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.083433:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.083454:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.093230:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70561792 -> 70565888 00002000:00010000:3.0:1789760756.093236:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137840 -> 137848 00010000:00010000:3.0:1789760756.093251:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.093264:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.093274:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.093281:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.093287:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.093297:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.093318:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.093326:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70561792->70565887] (req 70561792->70565887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.093528:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503590144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10199us (11086us total) trans 4294984534 rc 0/0 00000100:00100000:3.0:1789760756.093538:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17238 00000100:00100000:3.0:1789760756.093548:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686196543295 time 686207918443 state 0x2 ref 0 00000100:00100000:3.0:1789760756.097877:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503590272 00000100:00100000:3.0:1789760756.097888:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503590272 00000100:00100000:3.0:1789760756.097968:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686196543295 time 686207918443 state 0x0 ref 1 00000100:00100000:3.0:1789760756.097983:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686208178859 time 686207918443 state 0x1 ref 1 00000100:00100000:3.0:1789760756.098012:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686208178859 time 686222618955 state 0x0 ref 1 00000100:00100000:3.0:1789760756.098025:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503590272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.098073:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70565888, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a78 00010000:00010000:3.0:1789760756.098082:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.098097:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.098102:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.098108:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.098132:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.106467:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70565888 -> 70569984 00002000:00010000:3.0:1789760756.106473:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137848 -> 137856 00010000:00010000:3.0:1789760756.106494:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106511:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106519:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106526:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.106531:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106539:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106558:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.106566:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70565888->70569983] (req 70565888->70569983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.106674:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503590272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8645us (8929us total) trans 4294984535 rc 0/0 00000100:00100000:3.0:1789760756.106686:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17239 00000100:00100000:3.0:1789760756.106695:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686208178859 time 686222618955 state 0x2 ref 0 00000100:00100000:3.0:1789760756.141327:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503590656 00000100:00100000:3.0:1789760756.141337:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503590656 00000100:00100000:3.0:1789760756.141436:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686235746615 time 686247313582 state 0x0 ref 1 00000100:00100000:3.0:1789760756.141453:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686247573998 time 686247313582 state 0x1 ref 1 00000100:00100000:3.0:1789760756.141597:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686247573998 time 686266091519 state 0x0 ref 1 00000100:00100000:3.0:1789760756.141614:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503590656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.141660:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70578176, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a90 00010000:00010000:3.0:1789760756.141670:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.141701:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.141706:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.141712:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.141725:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.152248:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70578176 -> 70582272 00002000:00010000:3.0:1789760756.152253:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137872 -> 137880 00010000:00010000:3.0:1789760756.152458:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.152474:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.152579:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.152590:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.152595:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.152605:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.152626:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.152635:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70578176->70582271] (req 70578176->70582271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.152932:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503590656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11316us (11719us total) trans 4294984538 rc 0/0 00000100:00100000:3.0:1789760756.152943:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17242 00000100:00100000:3.0:1789760756.152954:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686247573998 time 686266091519 state 0x2 ref 0 00000100:00100000:3.0:1789760756.159713:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503590784 00000100:00100000:3.0:1789760756.159723:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503590784 00000100:00100000:3.0:1789760756.159790:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686247573998 time 686266091519 state 0x0 ref 1 00000100:00100000:3.0:1789760756.159855:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686266351935 time 686266091519 state 0x1 ref 1 00000100:00100000:3.0:1789760756.160589:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686266351935 time 686285193016 state 0x0 ref 1 00000100:00100000:3.0:1789760756.160608:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503590784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.160658:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70582272, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21a98 00010000:00010000:3.0:1789760756.160668:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.160684:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.160689:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.160695:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.160707:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.167055:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70582272 -> 70586368 00002000:00010000:3.0:1789760756.167061:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137880 -> 137888 00010000:00010000:3.0:1789760756.167074:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.167093:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.167102:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.167110:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.167114:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.167123:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.167461:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.167471:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70582272->70586367] (req 70582272->70586367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.167676:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503590784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7067us (8236us total) trans 4294984539 rc 0/0 00000100:00100000:3.0:1789760756.167687:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17243 00000100:00100000:3.0:1789760756.167697:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686266351935 time 686285193016 state 0x2 ref 0 00000100:00100000:3.0:1789760756.201216:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503591168 00000100:00100000:3.0:1789760756.201226:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503591168 00000100:00100000:3.0:1789760756.201294:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686295606163 time 686310337701 state 0x0 ref 1 00000100:00100000:3.0:1789760756.201309:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686310598117 time 686310337701 state 0x1 ref 1 00000100:00100000:3.0:1789760756.201383:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686310598117 time 686325988737 state 0x0 ref 1 00000100:00100000:3.0:1789760756.201434:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503591168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.201623:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70594560, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21ab0 00010000:00010000:3.0:1789760756.201634:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.201649:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.201654:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.201661:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.201676:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.207893:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70594560 -> 70598656 00002000:00010000:3.0:1789760756.207899:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137904 -> 137912 00010000:00010000:3.0:1789760756.207911:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.207923:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.207937:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.207945:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.207950:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.207959:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.207979:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.207987:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70594560->70598655] (req 70594560->70598655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.208104:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503591168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6667us (7016us total) trans 4294984542 rc 0/0 00000100:00100000:3.0:1789760756.208115:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17246 00000100:00100000:3.0:1789760756.208124:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686310598117 time 686325988737 state 0x2 ref 0 00000100:00100000:3.0:1789760756.212957:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503591296 00000100:00100000:3.0:1789760756.212967:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503591296 00000100:00100000:3.0:1789760756.213024:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686310598117 time 686325988737 state 0x0 ref 1 00000100:00100000:3.0:1789760756.213163:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686326249153 time 686325988737 state 0x1 ref 1 00000100:00100000:3.0:1789760756.213246:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686326249153 time 686337851358 state 0x0 ref 1 00000100:00100000:3.0:1789760756.213259:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503591296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.213330:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70598656, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21ab8 00010000:00010000:3.0:1789760756.213341:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.213357:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.213361:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.213368:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.213382:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.219775:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70598656 -> 70602752 00002000:00010000:3.0:1789760756.219780:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137912 -> 137920 00010000:00010000:3.0:1789760756.219797:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.219818:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.219827:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.219840:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.219845:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.219855:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.219876:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.219884:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70598656->70602751] (req 70598656->70602751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.220077:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503591296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6816us (7237us total) trans 4294984543 rc 0/0 00000100:00100000:3.0:1789760756.220088:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17247 00000100:00100000:3.0:1789760756.220097:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686326249153 time 686337851358 state 0x2 ref 0 00010000:00010000:3.0:1789760756.245452:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760756.254463:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503591680 00000100:00100000:3.0:1789760756.254474:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503591680 00000100:00100000:3.0:1789760756.254544:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686349253840 time 686363681970 state 0x0 ref 1 00000100:00100000:3.0:1789760756.254560:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686363942386 time 686363681970 state 0x1 ref 1 00000100:00100000:3.0:1789760756.254591:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686363942386 time 686379197732 state 0x0 ref 1 00000100:00100000:3.0:1789760756.254603:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503591680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.254652:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70610944, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21ad0 00010000:00010000:3.0:1789760756.254663:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.254685:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.254690:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.254696:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.254710:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.262425:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70610944 -> 70615040 00002000:00010000:3.0:1789760756.262431:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137936 -> 137944 00010000:00010000:3.0:1789760756.262444:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.262457:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.262466:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.262474:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.262737:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.262750:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.262783:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.262792:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edf6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70610944->70615039] (req 70610944->70615039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.262951:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503591680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8342us (8643us total) trans 4294984546 rc 0/0 00000100:00100000:3.0:1789760756.262962:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17250 00000100:00100000:3.0:1789760756.262971:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686363942386 time 686379197732 state 0x2 ref 0 00000100:00100000:3.0:1789760756.267945:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503591808 00000100:00100000:3.0:1789760756.267954:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503591808 00000100:00100000:3.0:1789760756.268008:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686363942386 time 686379197732 state 0x0 ref 1 00000100:00100000:3.0:1789760756.268023:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686379458148 time 686379197732 state 0x1 ref 1 00000100:00100000:3.0:1789760756.268056:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686379458148 time 686392662195 state 0x0 ref 1 00000100:00100000:3.0:1789760756.268068:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503591808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.268114:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70615040, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21ad8 00010000:00010000:3.0:1789760756.268123:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.268138:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.268142:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.268148:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.268161:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.273899:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70615040 -> 70619136 00002000:00010000:3.0:1789760756.273906:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137944 -> 137952 00010000:00010000:3.0:1789760756.273920:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.273933:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.273943:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.273950:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.273955:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.273964:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.273984:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.273992:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70615040->70619135] (req 70615040->70619135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.274137:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503591808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6066us (6280us total) trans 4294984547 rc 0/0 00000100:00100000:3.0:1789760756.274147:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17251 00000100:00100000:3.0:1789760756.274156:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686379458148 time 686392662195 state 0x2 ref 0 00000100:00100000:3.0:1789760756.303046:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503592192 00000100:00100000:3.0:1789760756.303055:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503592192 00000100:00100000:3.0:1789760756.303100:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686406337139 time 686414929470 state 0x0 ref 1 00000100:00100000:3.0:1789760756.303111:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686415189886 time 686414929470 state 0x1 ref 1 00000100:00100000:3.0:1789760756.303138:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686415189886 time 686427744580 state 0x0 ref 1 00000100:00100000:3.0:1789760756.303150:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503592192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.303198:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70627328, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21af0 00010000:00010000:3.0:1789760756.303206:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.303222:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.303239:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.303245:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.303257:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.311058:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70627328 -> 70631424 00002000:00010000:3.0:1789760756.311066:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137968 -> 137976 00010000:00010000:3.0:1789760756.311080:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.311095:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.311104:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.311112:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.311117:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.311127:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.311147:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.311161:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70627328->70631423] (req 70627328->70631423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.311281:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503592192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8127us (8311us total) trans 4294984550 rc 0/0 00000100:00100000:3.0:1789760756.311291:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17254 00000100:00100000:3.0:1789760756.311300:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686415189886 time 686427744580 state 0x2 ref 0 00000100:00100000:3.0:1789760756.316778:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503592320 00000100:00100000:3.0:1789760756.316788:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503592320 00000100:00100000:3.0:1789760756.316907:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686415189886 time 686427744580 state 0x0 ref 1 00000100:00100000:3.0:1789760756.317295:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686428004996 time 686427744580 state 0x1 ref 1 00000100:00100000:3.0:1789760756.317430:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686428004996 time 686442026943 state 0x0 ref 1 00000100:00100000:3.0:1789760756.317445:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503592320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.317508:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70631424, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21af8 00010000:00010000:3.0:1789760756.317520:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.317536:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.317541:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.317547:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.317565:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.336351:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70631424 -> 70635520 00002000:00010000:3.0:1789760756.336360:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 137976 -> 137984 00010000:00010000:3.0:1789760756.336374:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.336417:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.336429:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.336436:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.336441:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.336450:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.336470:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.336479:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edf8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70631424->70635519] (req 70631424->70635519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.336597:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503592320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19151us (19917us total) trans 4294984551 rc 0/0 00000100:00100000:3.0:1789760756.336608:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17255 00000100:00100000:3.0:1789760756.336618:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686428004996 time 686442026943 state 0x2 ref 0 00000001:00080000:3.0:1789760756.346682:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984552 is committed 00000100:00100000:3.0:1789760756.371194:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503592704 00000100:00100000:3.0:1789760756.371205:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503592704 00000100:00100000:3.0:1789760756.371260:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686465488489 time 686483360537 state 0x0 ref 1 00000100:00100000:3.0:1789760756.371275:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686483620953 time 686483360537 state 0x1 ref 1 00000100:00100000:3.0:1789760756.371308:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686483620953 time 686495915223 state 0x0 ref 1 00000100:00100000:3.0:1789760756.371320:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503592704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.371365:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70643712, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21b10 00010000:00010000:3.0:1789760756.371375:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.371419:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.371424:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.371431:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.371445:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.377299:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70643712 -> 70647808 00002000:00010000:3.0:1789760756.377305:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138000 -> 138008 00010000:00010000:3.0:1789760756.377386:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.377677:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.377687:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.377694:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.377700:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.377709:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.377728:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.377737:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083edfa2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70643712->70647807] (req 70643712->70647807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.377963:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503592704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6641us (6917us total) trans 4294984554 rc 0/0 00000100:00100000:3.0:1789760756.377975:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17258 00000100:00100000:3.0:1789760756.377985:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686483620953 time 686495915223 state 0x2 ref 0 00000100:00100000:3.0:1789760756.383457:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503592832 00000100:00100000:3.0:1789760756.383563:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503592832 00000100:00100000:3.0:1789760756.383619:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686483620953 time 686495915223 state 0x0 ref 1 00000100:00100000:3.0:1789760756.383635:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686496175639 time 686495915223 state 0x1 ref 1 00000100:00100000:3.0:1789760756.383708:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686496175639 time 686508313968 state 0x0 ref 1 00000100:00100000:3.0:1789760756.383723:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503592832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.383780:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70647808, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21b18 00010000:00010000:3.0:1789760756.383789:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.383815:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.383819:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.383824:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.383838:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.388939:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70647808 -> 70651904 00002000:00010000:3.0:1789760756.388945:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138008 -> 138016 00010000:00010000:3.0:1789760756.388959:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.388972:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.388981:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.388988:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.388993:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.389002:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.389023:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.389032:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083edfa9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70647808->70651903] (req 70647808->70651903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.389302:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503592832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5578us (6232us total) trans 4294984555 rc 0/0 00000100:00100000:3.0:1789760756.389313:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17259 00000100:00100000:3.0:1789760756.389323:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686496175639 time 686508313968 state 0x2 ref 0 00000100:00100000:3.0:1789760756.429775:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503593216 00000100:00100000:3.0:1789760756.429788:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503593216 00000100:00100000:3.0:1789760756.435996:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523599104 00000100:00100000:3.0:1789760756.436005:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523599104 00000100:00100000:3.0:1789760756.436062:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1789760756.436071:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523599104:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760756.436104:0:6518:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:3.0:1789760756.436160:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523599104:12345-0@lo:41:osp-pre-0-1.0 Request processed in 89us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.436167:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000001:00080000:3.0:1789760756.436216:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984558 is committed 00000100:00100000:3.0:1789760756.441701:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760756.450538:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984559 is committed 00000100:00100000:3.0:1789760756.459032:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760756.465771:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984560 is committed 00000100:00100000:3.0:1789760756.481991:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760756.562369:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503594368 00000100:00100000:3.0:1789760756.562384:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503594368 00000100:00100000:3.0:1789760756.632355:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503594880 00000100:00100000:3.0:1789760756.632364:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503594880 00000100:00100000:3.0:1789760756.632433:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686725807877 time 686743522791 state 0x0 ref 1 00000100:00100000:3.0:1789760756.632448:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686743783207 time 686743522791 state 0x1 ref 1 00000100:00100000:3.0:1789760756.632710:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686743783207 time 686757314805 state 0x0 ref 1 00000001:00080000:3.0:1789760756.637863:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984571 is committed 00000100:00100000:3.0:1789760756.649465:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760756.653732:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984572 is committed 00000001:00080000:3.0:1789760756.664809:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984573 is committed 00000100:00100000:3.0:1789760756.672716:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503595264 00000100:00100000:3.0:1789760756.672726:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503595264 00000100:00100000:3.0:1789760756.672786:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686774553759 time 686784813801 state 0x0 ref 1 00000100:00100000:3.0:1789760756.672801:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686785074217 time 686784813801 state 0x1 ref 1 00000100:00100000:3.0:1789760756.672831:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686785074217 time 686797437518 state 0x0 ref 1 00000100:00100000:3.0:1789760756.672843:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503595264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.672892:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70725632, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21bb0 00010000:00010000:3.0:1789760756.672901:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.672916:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.672920:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.672927:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.672941:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.676601:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70725632 -> 70729728 00002000:00010000:3.0:1789760756.676607:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138160 -> 138168 00010000:00010000:3.0:1789760756.676621:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676641:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676650:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676658:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.676663:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676672:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676691:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676699:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee02e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70725632->70729727] (req 70725632->70729727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.676828:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503595264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3980us (4251us total) trans 4294984574 rc 0/0 00000100:00100000:3.0:1789760756.676838:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17278 00000100:00100000:3.0:1789760756.676848:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686785074217 time 686797437518 state 0x2 ref 0 00000100:00100000:3.0:1789760756.680597:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760756.686122:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984575 is committed 00000100:00100000:3.0:1789760756.691187:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503595520 00000100:00100000:3.0:1789760756.691197:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503595520 00000100:00100000:3.0:1789760756.691261:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686797697934 time 686806071022 state 0x0 ref 1 00000100:00100000:3.0:1789760756.691275:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686806331438 time 686806071022 state 0x1 ref 1 00000100:00100000:3.0:1789760756.691303:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686806331438 time 686815910646 state 0x0 ref 1 00000100:00100000:3.0:1789760756.691315:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff519850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503595520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.691363:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70733824, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21bc0 00010000:00010000:3.0:1789760756.691372:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee03c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.691387:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.691420:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.691426:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee03c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70733824->70737919] (req 70733824->70737919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.691439:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee03c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70733824->70737919] (req 70733824->70737919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.710734:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760756.721015:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984577 is committed 00000100:00100000:3.0:1789760756.732726:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503595776 00000100:00100000:3.0:1789760756.732737:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503595776 00000100:00100000:3.0:1789760756.732807:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686816171062 time 686840073691 state 0x0 ref 1 00000100:00100000:3.0:1789760756.732825:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686840334107 time 686840073691 state 0x1 ref 1 00000100:00100000:3.0:1789760756.732855:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686840334107 time 686857461521 state 0x0 ref 1 00000100:00100000:3.0:1789760756.732867:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503595776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.732911:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70742016, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21bd0 00010000:00010000:3.0:1789760756.732920:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.732935:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.732940:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.732946:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.732961:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.738283:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70742016 -> 70746112 00002000:00010000:3.0:1789760756.738288:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138192 -> 138200 00010000:00010000:3.0:1789760756.738301:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.738314:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.738323:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.738331:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.738336:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.738344:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.738362:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.738370:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee04a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70742016->70746111] (req 70742016->70746111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.738751:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503595776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5880us (6273us total) trans 4294984578 rc 0/0 00000100:00100000:3.0:1789760756.738771:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17282 00000100:00100000:3.0:1789760756.738781:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686840334107 time 686857461521 state 0x2 ref 0 00000001:00080000:3.0:1789760756.748364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984579 is committed 00000001:00080000:3.0:1789760756.762885:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984580 is committed 00000100:00100000:3.0:1789760756.766470:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760756.775717:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984581 is committed 00000100:00100000:3.0:1789760756.785166:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503596288 00000100:00100000:3.0:1789760756.785176:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503596288 00000100:00100000:3.0:1789760756.785291:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686882829313 time 686891476613 state 0x0 ref 1 00000100:00100000:3.0:1789760756.785309:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686891737029 time 686891476613 state 0x1 ref 1 00000100:00100000:3.0:1789760756.785375:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686891737029 time 686909980328 state 0x0 ref 1 00000100:00100000:3.0:1789760756.785426:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503596288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.785474:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70758400, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x21bf0 00010000:00010000:3.0:1789760756.785491:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.785507:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.785511:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.785517:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.785529:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.789882:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70758400 -> 70762496 00002000:00010000:3.0:1789760756.789887:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:3.0:1789760756.789890:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00002000:00010000:3.0:1789760756.789892:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138224 -> 138232 00010000:00010000:3.0:1789760756.789922:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.789940:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.790018:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.790026:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.790032:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.790041:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.790061:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.790069:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee066 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70758400->70762495] (req 70758400->70762495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.791225:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503596288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5820us (6119us total) trans 4294984582 rc 0/0 00000100:00100000:3.0:1789760756.791239:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17286 00000100:00100000:3.0:1789760756.791250:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686891737029 time 686909980328 state 0x2 ref 0 00000100:00100000:3.0:1789760756.802222:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503596416 00000100:00100000:3.0:1789760756.802233:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503596416 00000100:00100000:3.0:1789760756.802285:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686891737029 time 686909980328 state 0x0 ref 1 00000100:00100000:3.0:1789760756.802301:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686910240744 time 686909980328 state 0x1 ref 1 00000100:00100000:3.0:1789760756.802344:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686910240744 time 686926950558 state 0x0 ref 1 00000100:00100000:3.0:1789760756.802357:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503596416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.802420:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70762496, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21bf8 00010000:00010000:3.0:1789760756.802430:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.802446:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.802450:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.802456:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.802470:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.807835:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70762496 -> 70766592 00002000:00010000:3.0:1789760756.807840:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138232 -> 138240 00010000:00010000:3.0:1789760756.807852:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.807865:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.807874:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.807886:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.807892:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.807900:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.807918:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.807926:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee06d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70762496->70766591] (req 70762496->70766591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.808023:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503596416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5664us (6237us total) trans 4294984583 rc 0/0 00000100:00100000:3.0:1789760756.808032:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17287 00000100:00100000:3.0:1789760756.808041:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686910240744 time 686926950558 state 0x2 ref 0 00000001:00080000:3.0:1789760756.829711:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984585 is committed 00000001:00080000:3.0:1789760756.841225:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984586 is committed 00000100:00100000:3.0:1789760756.847164:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503596928 00000100:00100000:3.0:1789760756.847174:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503596928 00000100:00100000:3.0:1789760756.847238:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686949901073 time 686960471564 state 0x0 ref 1 00000100:00100000:3.0:1789760756.847253:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686960731980 time 686960471564 state 0x1 ref 1 00000100:00100000:3.0:1789760756.847290:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 686960731980 time 686971892423 state 0x0 ref 1 00000100:00100000:3.0:1789760756.847303:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503596928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.847443:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70778880, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21c18 00010000:00010000:3.0:1789760756.847453:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee089 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.847469:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760756.847473:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.847573:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70778880->70782975] (req 70778880->70782975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.847590:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70778880->70782975] (req 70778880->70782975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.907036:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984590 is committed 00000001:00080000:3.0:1789760756.932655:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984591 is committed 00000100:00100000:3.0:1789760756.952486:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503597696 00000100:00100000:3.0:1789760756.952497:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503597696 00000100:00100000:3.0:1789760756.952553:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687047250049 time 687065397253 state 0x0 ref 1 00000100:00100000:3.0:1789760756.952568:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687065657669 time 687065397253 state 0x1 ref 1 00000100:00100000:3.0:1789760756.953437:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687065657669 time 687078037101 state 0x0 ref 1 00000100:00100000:3.0:1789760756.953455:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503597696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760756.953504:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70803456, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21c48 00010000:00010000:3.0:1789760756.953513:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.953530:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760756.953535:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.953542:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.953556:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.962053:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70803456 -> 70807552 00002000:00010000:3.0:1789760756.962060:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138312 -> 138320 00010000:00010000:3.0:1789760756.962072:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.962084:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.962093:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.962100:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760756.962105:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.962118:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.962137:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.962145:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70803456->70807551] (req 70803456->70807551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.962325:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503597696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8869us (10242us total) trans 4294984593 rc 0/0 00000100:00100000:3.0:1789760756.962336:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17297 00000100:00100000:3.0:1789760756.962345:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687065657669 time 687078037101 state 0x2 ref 0 00000100:00100000:3.0:1789760757.013112:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503598208 00000100:00100000:3.0:1789760757.013122:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503598208 00000100:00100000:3.0:1789760757.013176:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687110141613 time 687123304382 state 0x0 ref 1 00000100:00100000:3.0:1789760757.013191:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687123564798 time 687123304382 state 0x1 ref 1 00000100:00100000:3.0:1789760757.013223:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687123564798 time 687137829725 state 0x0 ref 1 00000100:00100000:3.0:1789760757.013235:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503598208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.013280:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70819840, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21c68 00010000:00010000:3.0:1789760757.013288:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.013304:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.013308:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.013314:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.013328:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.021347:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70819840 -> 70823936 00002000:00010000:3.0:1789760757.021353:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138344 -> 138352 00010000:00010000:3.0:1789760757.021379:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.021428:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.021437:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.021444:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.021449:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.021459:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.021493:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.021500:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70819840->70823935] (req 70819840->70823935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.021602:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503598208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8364us (8884us total) trans 4294984597 rc 0/0 00000100:00100000:3.0:1789760757.021613:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17301 00000100:00100000:3.0:1789760757.021623:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687123564798 time 687137829725 state 0x2 ref 0 00000100:00100000:3.0:1789760757.068592:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503598720 00000100:00100000:3.0:1789760757.068603:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503598720 00000100:00100000:3.0:1789760757.068659:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687161557988 time 687184063447 state 0x0 ref 1 00000100:00100000:3.0:1789760757.068673:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687184323863 time 687184063447 state 0x1 ref 1 00000100:00100000:3.0:1789760757.068694:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687184323863 time 687193301026 state 0x0 ref 1 00000100:00100000:3.0:1789760757.068706:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503598720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.068761:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70836224, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21c88 00010000:00010000:3.0:1789760757.068770:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.068785:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.068993:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.069001:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.069016:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.077010:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70836224 -> 70840320 00002000:00010000:3.0:1789760757.077018:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138376 -> 138384 00010000:00010000:3.0:1789760757.077033:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.077044:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.077053:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.077060:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.077065:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.077075:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.077094:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.077466:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee0eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70836224->70840319] (req 70836224->70840319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.077684:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503598720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8973us (9291us total) trans 4294984601 rc 0/0 00000100:00100000:3.0:1789760757.077695:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17305 00000100:00100000:3.0:1789760757.077705:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687184323863 time 687193301026 state 0x2 ref 0 00000100:00100000:3.0:1789760757.139036:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503599232 00000100:00100000:3.0:1789760757.139047:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503599232 00000100:00100000:3.0:1789760757.139101:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687227788894 time 687241990717 state 0x0 ref 1 00000100:00100000:3.0:1789760757.139116:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687242251133 time 687241990717 state 0x1 ref 1 00000100:00100000:3.0:1789760757.139149:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687242251133 time 687263755111 state 0x0 ref 1 00000100:00100000:3.0:1789760757.139163:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503599232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.139209:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70852608, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21ca8 00010000:00010000:3.0:1789760757.139220:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.139236:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.139241:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.139247:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.139266:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.145281:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70852608 -> 70856704 00002000:00010000:3.0:1789760757.145286:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138408 -> 138416 00010000:00010000:3.0:1789760757.145298:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.145310:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.145319:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.145326:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.145331:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.145340:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.145359:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.145367:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee107 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70852608->70856703] (req 70852608->70856703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.145525:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503599232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6359us (6671us total) trans 4294984605 rc 0/0 00000100:00100000:3.0:1789760757.145535:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17309 00000100:00100000:3.0:1789760757.145544:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687242251133 time 687263755111 state 0x2 ref 0 00000100:00100000:3.0:1789760757.190289:0:6505:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503599744 00000100:00100000:3.0:1789760757.190303:0:6505:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503599744 00000100:00100000:3.0:1789760757.190458:0:6505:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 480 00000100:00100000:3.0:1789760757.190470:0:6505:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dea8aac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2390:x1876699503599744:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.190558:0:6505:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dea8aac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2390:x1876699503599744:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 90us (1328us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.190565:0:6505:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 480 00000100:00100000:3.0:1789760757.193851:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503599872 00000100:00100000:3.0:1789760757.193861:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503599872 00000100:00100000:3.0:1789760757.193920:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 921 00000100:00100000:3.0:1789760757.193929:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2390:x1876699503599872:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.193995:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2390:x1876699503599872:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 67us (690us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.194002:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 921 00000100:00080000:3.0:1789760757.194024:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760757.194065:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.194975:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.194999:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.195020:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503600000 00000100:00100000:3.0:1789760757.195026:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503600000 00000100:00100000:3.0:1789760757.195058:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503600256 00000100:00100000:3.0:1789760757.195060:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503600256 00000100:00100000:3.0:1789760757.195074:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503600128 00000100:00100000:3.0:1789760757.195076:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503600128 00000100:00100000:3.0:1789760757.195089:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 922 00000100:00100000:3.0:1789760757.195096:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca080700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503600000:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.195134:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca080700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503600000:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 39us (1070us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.195140:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 922 00000100:00100000:3.0:1789760757.195154:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 923 00000100:00100000:3.0:1789760757.195159:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca081180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503600256:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.195180:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca081180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503600256:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 22us (211us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.195186:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 923 00000100:00100000:3.0:1789760757.195196:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 924 00000100:00100000:3.0:1789760757.195201:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca081c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503600128:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.195216:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca081c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503600128:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 16us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.195222:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 924 00000100:00080000:3.0:1789760757.195238:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760757.195240:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760757.197614:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503600384 00000100:00100000:3.0:1789760757.197631:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503600384 00000100:00100000:3.0:1789760757.198726:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687294675164 time 687304714599 state 0x0 ref 1 00000100:00100000:3.0:1789760757.198742:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687304975015 time 687304714599 state 0x1 ref 1 00000100:00100000:3.0:1789760757.199000:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687304975015 time 687323604982 state 0x0 ref 1 00000100:00100000:3.0:1789760757.199021:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503600384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.199069:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70868992, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21cc8 00010000:00010000:3.0:1789760757.199079:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.199094:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.199099:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.199105:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.199118:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.207173:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70868992 -> 70873088 00002000:00010000:3.0:1789760757.207182:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138440 -> 138448 00010000:00010000:3.0:1789760757.207199:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.207211:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.207220:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.207233:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.207238:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.207248:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.207267:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.207274:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee123 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70868992->70873087] (req 70868992->70873087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.207624:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503600384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8376us (11467us total) trans 4294984609 rc 0/0 00000100:00100000:3.0:1789760757.207636:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17313 00000100:00100000:3.0:1789760757.207646:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687304975015 time 687323604982 state 0x2 ref 0 00000100:00100000:3.0:1789760757.248730:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503600896 00000100:00100000:3.0:1789760757.248740:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503600896 00000100:00100000:3.0:1789760757.248813:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687348098092 time 687363634336 state 0x0 ref 1 00000100:00100000:3.0:1789760757.248829:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687363894752 time 687363634336 state 0x1 ref 1 00000100:00100000:3.0:1789760757.248858:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687363894752 time 687373465131 state 0x0 ref 1 00000100:00100000:3.0:1789760757.248873:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503600896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.248920:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70885376, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21ce8 00010000:00010000:3.0:1789760757.248930:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.248945:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.248950:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.248955:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.248969:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.254297:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70885376 -> 70889472 00002000:00010000:3.0:1789760757.254303:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138472 -> 138480 00010000:00010000:3.0:1789760757.254316:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.254329:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.254338:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.254345:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.254350:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.254359:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.254379:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.254386:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee13f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70885376->70889471] (req 70885376->70889471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.254548:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503600896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5670us (5969us total) trans 4294984613 rc 0/0 00000100:00100000:3.0:1789760757.254559:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17317 00000100:00100000:3.0:1789760757.254568:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687363894752 time 687373465131 state 0x2 ref 0 00010000:00010000:3.0:1789760757.266714:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760757.275864:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984615 is committed 00000100:00100000:3.0:1789760757.280058:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503601280 00000100:00100000:3.0:1789760757.280067:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503601280 00000100:00100000:3.0:1789760757.280122:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687383383286 time 687392741329 state 0x0 ref 1 00000100:00100000:3.0:1789760757.280137:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687393001745 time 687392741329 state 0x1 ref 1 00000100:00100000:3.0:1789760757.280169:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687393001745 time 687404775464 state 0x0 ref 1 00000100:00100000:3.0:1789760757.280182:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503601280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.280232:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70897664, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21d00 00010000:00010000:3.0:1789760757.280243:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee154 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.280259:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.280263:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.280269:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70897664->70901759] (req 70897664->70901759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.280284:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70897664->70901759] (req 70897664->70901759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.291163:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503601408 00000100:00100000:3.0:1789760757.291172:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503601408 00000100:00100000:3.0:1789760757.291226:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687393001745 time 687404775464 state 0x0 ref 1 00000100:00100000:3.0:1789760757.291240:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687405035880 time 687404775464 state 0x1 ref 1 00000100:00100000:3.0:1789760757.291270:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687405035880 time 687415876841 state 0x0 ref 1 00000100:00100000:3.0:1789760757.291282:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503601408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.291328:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70901760, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21d08 00010000:00010000:3.0:1789760757.291338:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee15b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.291353:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.291358:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.291364:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee15b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70901760->70905855] (req 70901760->70905855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.291378:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee15b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70901760->70905855] (req 70901760->70905855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.297595:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984617 is committed 00000001:00080000:3.0:1789760757.306233:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984618 is committed 00000001:00080000:3.0:1789760757.314787:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984619 is committed 00000100:00100000:3.0:1789760757.334276:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503601920 00000100:00100000:3.0:1789760757.334287:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503601920 00000100:00100000:3.0:1789760757.334342:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687435205685 time 687447879239 state 0x0 ref 1 00000100:00100000:3.0:1789760757.334358:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687448139655 time 687447879239 state 0x1 ref 1 00000100:00100000:3.0:1789760757.334416:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687448139655 time 687458995484 state 0x0 ref 1 00000100:00100000:3.0:1789760757.334430:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503601920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.334784:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70918144, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21d28 00010000:00010000:3.0:1789760757.334802:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee177 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.334817:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.334821:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.334827:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70918144->70922239] (req 70918144->70922239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.334846:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70918144->70922239] (req 70918144->70922239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.342215:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984621 is committed 00000100:00100000:3.0:1789760757.392180:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503602432 00000100:00100000:3.0:1789760757.392191:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503602432 00000100:00100000:3.0:1789760757.392249:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687487738092 time 687501564917 state 0x0 ref 1 00000100:00100000:3.0:1789760757.392265:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687501825333 time 687501564917 state 0x1 ref 1 00000100:00100000:3.0:1789760757.392299:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687501825333 time 687516905118 state 0x0 ref 1 00000100:00100000:3.0:1789760757.392310:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503602432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.392358:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70934528, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21d48 00010000:00010000:3.0:1789760757.392367:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.392383:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.392416:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.392423:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.392438:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.401121:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 70934528 -> 70938624 00002000:00010000:3.0:1789760757.401126:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138568 -> 138576 00010000:00010000:3.0:1789760757.401140:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.401153:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.401162:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.401169:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.401174:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.401183:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.401203:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.401210:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee193 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70934528->70938623] (req 70934528->70938623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.401340:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503602432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9026us (9260us total) trans 4294984625 rc 0/0 00000100:00100000:3.0:1789760757.401352:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17329 00000100:00100000:3.0:1789760757.401363:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687501825333 time 687516905118 state 0x2 ref 0 00000100:00100000:3.0:1789760757.443785:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503602944 00000100:00100000:3.0:1789760757.443795:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503602944 00000100:00100000:3.0:1789760757.443856:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687545024449 time 687557459935 state 0x0 ref 1 00000100:00100000:3.0:1789760757.443872:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687557720351 time 687557459935 state 0x1 ref 1 00000100:00100000:3.0:1789760757.443987:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687557720351 time 687568591938 state 0x0 ref 1 00000100:00100000:3.0:1789760757.444000:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503602944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.444052:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 70950912, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21d68 00010000:00010000:3.0:1789760757.444061:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee1af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.444077:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.444081:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.444086:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee1af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70950912->70955007] (req 70950912->70955007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.444099:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee1af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [70950912->70955007] (req 70950912->70955007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.481961:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.497034:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.534990:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.546640:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.558634:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.597782:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.633879:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760757.645422:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503604736 00000100:00100000:3.0:1789760757.645434:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503604736 00000100:00100000:3.0:1789760757.645497:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687746604428 time 687758687727 state 0x0 ref 1 00000100:00100000:3.0:1789760757.645513:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687758948143 time 687758687727 state 0x1 ref 1 00000100:00100000:3.0:1789760757.645540:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687758948143 time 687770146248 state 0x0 ref 1 00000100:00100000:3.0:1789760757.645552:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503604736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.645598:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71008256, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21dd8 00010000:00010000:3.0:1789760757.645607:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee211 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.645623:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.645627:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.645633:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71008256->71012351] (req 71008256->71012351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.645647:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71008256->71012351] (req 71008256->71012351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.650135:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984643 is committed 00000001:00080000:3.0:1789760757.658959:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984644 is committed 00000100:00100000:3.0:1789760757.662151:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760757.670615:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984645 is committed 00000100:00100000:3.0:1789760757.683910:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503605120 00000100:00100000:3.0:1789760757.683921:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503605120 00000100:00100000:3.0:1789760757.683973:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687779308723 time 687787643116 state 0x0 ref 1 00000100:00100000:3.0:1789760757.683988:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687787903532 time 687787643116 state 0x1 ref 1 00000100:00100000:3.0:1789760757.684021:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687787903532 time 687808628262 state 0x0 ref 1 00000100:00100000:3.0:1789760757.684033:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503605120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.684081:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71020544, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21df0 00010000:00010000:3.0:1789760757.684090:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.684104:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.684109:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.684116:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.684129:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.689376:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71020544 -> 71024640 00002000:00010000:3.0:1789760757.689381:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138736 -> 138744 00010000:00010000:3.0:1789760757.689412:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.689426:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.689435:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.689443:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.689447:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.689457:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.689544:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.689553:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee226 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71020544->71024639] (req 71020544->71024639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.689735:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503605120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5698us (6026us total) trans 4294984646 rc 0/0 00000100:00100000:3.0:1789760757.689756:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17350 00000100:00100000:3.0:1789760757.689767:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687787903532 time 687808628262 state 0x2 ref 0 00002000:00010000:3.0:1789760757.709092:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71024640 -> 71028736 00002000:00010000:3.0:1789760757.709098:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138744 -> 138752 00010000:00010000:3.0:1789760757.709111:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ee22d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71024640->71028735] (req 71024640->71028735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.709123:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ee22d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71024640->71028735] (req 71024640->71028735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.709132:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ee22d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71024640->71028735] (req 71024640->71028735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.709141:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.709146:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ee22d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71024640->71028735] (req 71024640->71028735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.709155:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ee22d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71024640->71028735] (req 71024640->71028735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.709173:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ee22d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71024640->71028735] (req 71024640->71028735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.709180:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ee22d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71024640->71028735] (req 71024640->71028735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.709324:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503605248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15390us (15625us total) trans 4294984647 rc 0/0 00000100:00100000:3.0:1789760757.709335:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17351 00000100:00100000:3.0:1789760757.709346:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687808888678 time 687818523529 state 0x2 ref 0 00000100:00100000:3.0:1789760757.751590:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503605632 00000100:00100000:3.0:1789760757.751600:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503605632 00000100:00100000:3.0:1789760757.751658:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687838419483 time 687854638484 state 0x0 ref 1 00000100:00100000:3.0:1789760757.751673:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687854898900 time 687854638484 state 0x1 ref 1 00000100:00100000:3.0:1789760757.751699:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687854898900 time 687876306262 state 0x0 ref 1 00000100:00100000:3.0:1789760757.751710:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503605632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.751762:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71036928, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x21e10 00010000:00010000:3.0:1789760757.751772:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751787:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.751791:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.751798:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751811:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.758802:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71036928 -> 71041024 00002000:00010000:3.0:1789760757.758808:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138768 -> 138776 00010000:00010000:3.0:1789760757.758821:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.758838:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.758847:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.758855:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.758860:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.758868:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.758887:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.758894:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee242 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71036928->71041023] (req 71036928->71041023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.758991:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503605632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7276us (7602us total) trans 4294984650 rc 0/0 00000100:00100000:3.0:1789760757.759001:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17354 00000100:00100000:3.0:1789760757.759011:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687854898900 time 687876306262 state 0x2 ref 0 00000100:00100000:3.0:1789760757.764078:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760757.805680:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984653 is committed 00000100:00100000:3.0:1789760757.811015:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503606144 00000100:00100000:3.0:1789760757.811024:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503606144 00000100:00100000:3.0:1789760757.811076:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687907351560 time 687921681612 state 0x0 ref 1 00000100:00100000:3.0:1789760757.811102:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687921942028 time 687921681612 state 0x1 ref 1 00000100:00100000:3.0:1789760757.811132:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687921942028 time 687935739131 state 0x0 ref 1 00000100:00100000:3.0:1789760757.811145:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503606144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.811190:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71053312, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21e30 00010000:00010000:3.0:1789760757.811199:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.811213:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.811218:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.811224:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.811238:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.820660:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71053312 -> 71057408 00002000:00010000:3.0:1789760757.820669:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138800 -> 138808 00010000:00010000:3.0:1789760757.820683:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.820702:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.820710:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.820718:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.820722:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.820731:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.820762:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.820770:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee25e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71053312->71057407] (req 71053312->71057407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.821515:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503606144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10364us (10679us total) trans 4294984654 rc 0/0 00000100:00100000:3.0:1789760757.821526:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17358 00000100:00100000:3.0:1789760757.821535:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687921942028 time 687935739131 state 0x2 ref 0 00000100:00100000:3.0:1789760757.829564:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503606272 00000100:00100000:3.0:1789760757.829575:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503606272 00000100:00100000:3.0:1789760757.829633:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687921942028 time 687935739131 state 0x0 ref 1 00000100:00100000:3.0:1789760757.829649:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687935999547 time 687935739131 state 0x1 ref 1 00000100:00100000:3.0:1789760757.829689:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687935999547 time 687954295045 state 0x0 ref 1 00000100:00100000:3.0:1789760757.829702:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503606272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.829755:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71057408, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21e38 00010000:00010000:3.0:1789760757.829764:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.829780:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760757.829784:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.829797:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.829810:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.836621:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71057408 -> 71061504 00002000:00010000:3.0:1789760757.836627:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138808 -> 138816 00010000:00010000:3.0:1789760757.836640:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.836653:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.836668:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.836687:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760757.836692:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.836702:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.836722:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.836729:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee265 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71057408->71061503] (req 71057408->71061503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.851520:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984656 is committed 00000100:00100000:3.0:1789760757.882419:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503606656 00000100:00100000:3.0:1789760757.882429:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503606656 00000100:00100000:3.0:1789760757.882600:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687967236071 time 687986566708 state 0x0 ref 1 00000100:00100000:3.0:1789760757.882615:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687986827124 time 687986566708 state 0x1 ref 1 00000100:00100000:3.0:1789760757.882742:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 687986827124 time 688007346830 state 0x0 ref 1 00000100:00100000:3.0:1789760757.882764:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503606656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760757.883308:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71069696, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21e50 00010000:00010000:3.0:1789760757.883319:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee27a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.883345:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760757.883349:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.883355:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee27a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71069696->71073791] (req 71069696->71073791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.883371:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee27a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71069696->71073791] (req 71069696->71073791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.920118:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984660 is committed 00000001:00080000:3.0:1789760757.950650:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984662 is committed 00000001:00080000:3.0:1789760757.967647:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984663 is committed 00000001:00080000:3.0:1789760757.984998:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984664 is committed 00000001:00080000:3.0:1789760758.010832:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984665 is committed 00000100:00100000:3.0:1789760758.021604:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503607680 00000100:00100000:3.0:1789760758.021616:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503607680 00000100:00100000:3.0:1789760758.021696:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688100786977 time 688116257811 state 0x0 ref 1 00000100:00100000:3.0:1789760758.021711:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688116518227 time 688116257811 state 0x1 ref 1 00000100:00100000:3.0:1789760758.021756:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688116518227 time 688146353500 state 0x0 ref 1 00000100:00100000:3.0:1789760758.021777:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503607680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.021833:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71102464, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21e90 00010000:00010000:3.0:1789760758.021843:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.021858:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.021862:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.021869:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.021882:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.033462:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71102464 -> 71106560 00002000:00010000:3.0:1789760758.033475:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138896 -> 138904 00010000:00010000:3.0:1789760758.033489:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.033503:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.033513:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.033521:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.033526:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.033537:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.033558:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.033567:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71102464->71106559] (req 71102464->71106559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.033731:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503607680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11950us (12197us total) trans 4294984666 rc 0/0 00000100:00100000:3.0:1789760758.033742:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17370 00000100:00100000:3.0:1789760758.033762:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688116518227 time 688146353500 state 0x2 ref 0 00000100:00100000:3.0:1789760758.097851:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503608192 00000100:00100000:3.0:1789760758.097861:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503608192 00000100:00100000:3.0:1789760758.097913:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688190515155 time 688203366262 state 0x0 ref 1 00000100:00100000:3.0:1789760758.097928:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688203626678 time 688203366262 state 0x1 ref 1 00000100:00100000:3.0:1789760758.097960:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688203626678 time 688222566820 state 0x0 ref 1 00000100:00100000:3.0:1789760758.097972:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503608192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.098026:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71118848, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21eb0 00010000:00010000:3.0:1789760758.098037:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.098062:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.098067:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.098074:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.098088:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.105221:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71118848 -> 71122944 00002000:00010000:3.0:1789760758.105226:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138928 -> 138936 00010000:00010000:3.0:1789760758.105238:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.105251:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.105259:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.105267:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.105271:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.105280:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.105298:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.105310:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71118848->71122943] (req 71118848->71122943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.105478:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503608192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7496us (7697us total) trans 4294984670 rc 0/0 00000100:00100000:3.0:1789760758.105489:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17374 00000100:00100000:3.0:1789760758.105499:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688203626678 time 688222566820 state 0x2 ref 0 00000100:00100000:3.0:1789760758.161145:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503608704 00000100:00100000:3.0:1789760758.161155:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503608704 00000100:00100000:3.0:1789760758.161209:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688246091207 time 688272750971 state 0x0 ref 1 00000100:00100000:3.0:1789760758.161224:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688273011387 time 688272750971 state 0x1 ref 1 00000100:00100000:3.0:1789760758.161285:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688273011387 time 688285890099 state 0x0 ref 1 00000100:00100000:3.0:1789760758.161298:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503608704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.161344:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71135232, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21ed0 00010000:00010000:3.0:1789760758.161354:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.161369:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.161373:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.161379:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.161414:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.166769:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71135232 -> 71139328 00002000:00010000:3.0:1789760758.166774:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138960 -> 138968 00010000:00010000:3.0:1789760758.166891:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166905:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166913:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166921:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.166932:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166941:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166961:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.166968:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee2ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71135232->71139327] (req 71135232->71139327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.167134:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503608704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5831us (6141us total) trans 4294984674 rc 0/0 00000100:00100000:3.0:1789760758.167144:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17378 00000100:00100000:3.0:1789760758.167154:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688273011387 time 688285890099 state 0x2 ref 0 00000100:00100000:3.0:1789760758.216441:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503609216 00000100:00100000:3.0:1789760758.216451:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503609216 00000100:00100000:3.0:1789760758.216513:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688316925647 time 688329069864 state 0x0 ref 1 00000100:00100000:3.0:1789760758.216527:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688329330280 time 688329069864 state 0x1 ref 1 00000100:00100000:3.0:1789760758.216552:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688329330280 time 688341158662 state 0x0 ref 1 00000100:00100000:3.0:1789760758.216564:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503609216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.216612:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71151616, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21ef0 00010000:00010000:3.0:1789760758.216621:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.216636:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.216640:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.216646:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.216659:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.222740:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71151616 -> 71155712 00002000:00010000:3.0:1789760758.222757:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 138992 -> 139000 00010000:00010000:3.0:1789760758.222778:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.222982:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.222991:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.222999:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.223004:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.223013:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.223039:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.223048:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee306 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71151616->71155711] (req 71151616->71155711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.223343:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503609216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6775us (7030us total) trans 4294984678 rc 0/0 00000100:00100000:3.0:1789760758.223356:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17382 00000100:00100000:3.0:1789760758.223366:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688329330280 time 688341158662 state 0x2 ref 0 00000100:00100000:3.0:1789760758.279868:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503609728 00000100:00100000:3.0:1789760758.279878:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503609728 00000100:00100000:3.0:1789760758.279932:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688372206481 time 688387490076 state 0x0 ref 1 00000100:00100000:3.0:1789760758.279958:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688387750492 time 688387490076 state 0x1 ref 1 00000100:00100000:3.0:1789760758.279987:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688387750492 time 688404594291 state 0x0 ref 1 00000100:00100000:3.0:1789760758.280000:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503609728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.280044:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71168000, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21f10 00010000:00010000:3.0:1789760758.280053:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.280067:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.280071:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.280078:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.280091:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.287216:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71168000 -> 71172096 00002000:00010000:3.0:1789760758.287222:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139024 -> 139032 00010000:00010000:3.0:1789760758.287249:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287261:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287270:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287278:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.287283:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287291:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287310:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.287317:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee322 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71168000->71172095] (req 71168000->71172095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.287997:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503609728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7993us (8191us total) trans 4294984682 rc 0/0 00000100:00100000:3.0:1789760758.288008:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17386 00000100:00100000:3.0:1789760758.288019:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688387750492 time 688404594291 state 0x2 ref 0 00010000:00010000:3.0:1789760758.290627:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760758.335788:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503610240 00000100:00100000:3.0:1789760758.335799:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503610240 00000100:00100000:3.0:1789760758.335850:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688427670238 time 688443208987 state 0x0 ref 1 00000100:00100000:3.0:1789760758.335866:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688443469403 time 688443208987 state 0x1 ref 1 00000100:00100000:3.0:1789760758.335897:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688443469403 time 688460503317 state 0x0 ref 1 00000100:00100000:3.0:1789760758.335909:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503610240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.335952:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71184384, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21f30 00010000:00010000:3.0:1789760758.335961:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.335977:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.335982:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.335988:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.336002:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.341212:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71184384 -> 71188480 00002000:00010000:3.0:1789760758.341217:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139056 -> 139064 00010000:00010000:3.0:1789760758.341239:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341256:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341266:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341273:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.341278:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341287:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341307:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.341315:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee33e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71184384->71188479] (req 71184384->71188479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.341621:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503610240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5553us (5909us total) trans 4294984686 rc 0/0 00000100:00100000:3.0:1789760758.341634:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17390 00000100:00100000:3.0:1789760758.341644:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688443469403 time 688460503317 state 0x2 ref 0 00000100:00100000:3.0:1789760758.397624:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760758.468538:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503611264 00000100:00100000:3.0:1789760758.468548:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503611264 00000100:00100000:3.0:1789760758.468604:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688555070857 time 688575531521 state 0x0 ref 1 00000100:00100000:3.0:1789760758.468620:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688575791937 time 688575531521 state 0x1 ref 1 00000100:00100000:3.0:1789760758.468652:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688575791937 time 688593258064 state 0x0 ref 1 00000100:00100000:3.0:1789760758.468665:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503611264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.468711:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71217152, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21f70 00010000:00010000:3.0:1789760758.468721:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.468735:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.468750:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.468758:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.468779:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.475813:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71217152 -> 71221248 00002000:00010000:3.0:1789760758.475818:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139120 -> 139128 00010000:00010000:3.0:1789760758.475831:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.475843:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.475853:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.475860:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.475864:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.475874:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.475894:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.475902:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee376 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71217152->71221247] (req 71217152->71221247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.476017:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503611264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7349us (7574us total) trans 4294984694 rc 0/0 00000100:00100000:3.0:1789760758.476025:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17398 00000100:00100000:3.0:1789760758.476033:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688575791937 time 688593258064 state 0x2 ref 0 00000100:00100000:3.0:1789760758.484225:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760758.484247:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc8f71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523599232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760758.484311:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.484655:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760758.485089:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8f71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523599232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760758.485917:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523600128 00000100:00100000:3.0:1789760758.485922:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523600128 00000100:00100000:3.0:1789760758.485953:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523600640 00000100:00100000:3.0:1789760758.485956:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523600640 00000100:00100000:3.0:1789760758.485968:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523599488 00000100:00100000:3.0:1789760758.485970:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523599488 00000100:00100000:3.0:1789760758.485981:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523600000 00000100:00100000:3.0:1789760758.485983:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523600000 00000100:00100000:3.0:1789760758.486001:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523600512 00000100:00100000:3.0:1789760758.486004:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523600512 00000100:00100000:3.0:1789760758.486023:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1789760758.486030:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3637:x1876699523600128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.486082:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3637:x1876699523600128:12345-0@lo:400:kworker.0 Request processed in 52us (708us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.486090:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1789760758.486106:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1789760758.486113:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3637:x1876699523600640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.486144:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3637:x1876699523600640:12345-0@lo:400:kworker.0 Request processed in 32us (697us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.486151:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1789760758.486164:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 927 00000100:00100000:3.0:1789760758.486170:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3637:x1876699523599488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.486196:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3637:x1876699523599488:12345-0@lo:400:kworker.0 Request processed in 26us (675us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.486201:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 927 00000100:00100000:3.0:1789760758.486212:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 928 00000100:00100000:3.0:1789760758.486217:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3637:x1876699523600000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.486241:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3637:x1876699523600000:12345-0@lo:400:kworker.0 Request processed in 24us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.486246:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 928 00000100:00100000:3.0:1789760758.486256:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 929 00000100:00100000:3.0:1789760758.486261:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3637:x1876699523600512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.486285:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3637:x1876699523600512:12345-0@lo:400:kworker.0 Request processed in 24us (670us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.486291:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 929 00000100:00080000:3.0:1789760758.486314:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760758.486317:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760758.486326:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc743fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:afc717a1-11ba-4ef0-ab10-103aa692ca98:3639:1876699523599360:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.486356:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.486411:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc743ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523599872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.486436:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.486454:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ccea0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523600384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.488078:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.488101:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5e01be2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523600896:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760758.488127:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.488167:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc743fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:afc717a1-11ba-4ef0-ab10-103aa692ca98:3639:1876699523599360:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.488181:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc743ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523599872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.488212:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523600896 00000100:00100000:3.0:1789760758.488215:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523600896 00000100:00100000:3.0:1789760758.488238:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:3.0:1789760758.488244:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f73800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3639:x1876699523600896:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760758.488263:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760758.488296:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f73800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3639:x1876699523600896:12345-0@lo:13:osp-pre-1-1.0 Request processed in 52us (169us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.488302:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:3.0:1789760758.488334:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5e01be2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523600896:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760758.488376:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc743f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3638:1876699523599744:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.488386:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523600256:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.495627:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523600384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.515325:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503611648 00000100:00100000:3.0:1789760758.515336:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503611648 00000100:00100000:3.0:1789760758.515428:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688610307629 time 688628704222 state 0x0 ref 1 00000100:00100000:3.0:1789760758.515445:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688628964638 time 688628704222 state 0x1 ref 1 00000100:00100000:3.0:1789760758.529297:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760758.535481:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984698 is committed 00000100:00100000:3.0:1789760758.544204:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503611904 00000100:00100000:3.0:1789760758.544216:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503611904 00000100:00100000:3.0:1789760758.544281:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688640625188 time 688654235237 state 0x0 ref 1 00000100:00100000:3.0:1789760758.544298:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688654495653 time 688654235237 state 0x1 ref 1 00000100:00100000:3.0:1789760758.544320:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688654495653 time 688668926593 state 0x0 ref 1 00000100:00100000:3.0:1789760758.544332:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503611904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.544509:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71237632, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21f98 00010000:00010000:3.0:1789760758.544519:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.544535:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.544540:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.544546:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.544561:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.552294:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71237632 -> 71241728 00002000:00010000:3.0:1789760758.552300:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139160 -> 139168 00010000:00010000:3.0:1789760758.552313:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.552324:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.552333:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.552341:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.552346:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.552359:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.552379:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.552386:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee399 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71237632->71241727] (req 71237632->71241727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.552545:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503611904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8210us (8561us total) trans 4294984699 rc 0/0 00000100:00100000:3.0:1789760758.552561:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17403 00000100:00100000:3.0:1789760758.552571:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688654495653 time 688668926593 state 0x2 ref 0 00000100:00100000:3.0:1789760758.595981:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503612288 00000100:00100000:3.0:1789760758.595991:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503612288 00000100:00100000:3.0:1789760758.596041:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688682658669 time 688694711238 state 0x0 ref 1 00000100:00100000:3.0:1789760758.596057:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688694971654 time 688694711238 state 0x1 ref 1 00000100:00100000:3.0:1789760758.596087:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688694971654 time 688720693664 state 0x0 ref 1 00000100:00100000:3.0:1789760758.596100:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503612288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.596145:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71249920, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21fb0 00010000:00010000:3.0:1789760758.596153:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.596167:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.596172:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.596178:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.596191:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.602150:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71249920 -> 71254016 00002000:00010000:3.0:1789760758.602155:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139184 -> 139192 00010000:00010000:3.0:1789760758.602167:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.602179:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.602189:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.602196:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.602201:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.602210:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.602234:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.602241:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71249920->71254015] (req 71249920->71254015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.602359:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503612288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6254us (6551us total) trans 4294984702 rc 0/0 00000100:00100000:3.0:1789760758.602373:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17406 00000100:00100000:3.0:1789760758.602385:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688694971654 time 688720693664 state 0x2 ref 0 00000100:00100000:3.0:1789760758.611775:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503612416 00000100:00100000:3.0:1789760758.611803:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503612416 00000100:00100000:3.0:1789760758.611879:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688694971654 time 688720693664 state 0x0 ref 1 00000100:00100000:3.0:1789760758.611897:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688720954080 time 688720693664 state 0x1 ref 1 00000100:00100000:3.0:1789760758.611928:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688720954080 time 688736534916 state 0x0 ref 1 00000100:00100000:3.0:1789760758.611942:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503612416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.612003:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71254016, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21fb8 00010000:00010000:3.0:1789760758.612013:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee3b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.612031:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.612037:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.612043:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee3b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71254016->71258111] (req 71254016->71258111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.612059:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee3b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71254016->71258111] (req 71254016->71258111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.658659:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503612800 00000100:00100000:3.0:1789760758.658669:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503612800 00000100:00100000:3.0:1789760758.658722:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688754297462 time 688763654520 state 0x0 ref 1 00000100:00100000:3.0:1789760758.658737:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688763914936 time 688763654520 state 0x1 ref 1 00000100:00100000:3.0:1789760758.658776:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688763914936 time 688783383266 state 0x0 ref 1 00000100:00100000:3.0:1789760758.658798:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503612800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.658844:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71266304, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21fd0 00010000:00010000:3.0:1789760758.658852:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.658867:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.658872:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.658878:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.658892:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.667359:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71266304 -> 71270400 00002000:00010000:3.0:1789760758.667369:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139216 -> 139224 00010000:00010000:3.0:1789760758.667383:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.667568:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.667578:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.667587:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.667592:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.667602:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.667623:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.667631:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71266304->71270399] (req 71266304->71270399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.667808:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503612800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9007us (9207us total) trans 4294984706 rc 0/0 00000100:00100000:3.0:1789760758.667819:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17410 00000100:00100000:3.0:1789760758.667830:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688763914936 time 688783383266 state 0x2 ref 0 00000100:00100000:3.0:1789760758.729977:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503613312 00000100:00100000:3.0:1789760758.729988:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503613312 00000100:00100000:3.0:1789760758.730045:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688816546877 time 688833181209 state 0x0 ref 1 00000100:00100000:3.0:1789760758.730060:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688833441625 time 688833181209 state 0x1 ref 1 00000100:00100000:3.0:1789760758.730091:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688833441625 time 688854697217 state 0x0 ref 1 00000100:00100000:3.0:1789760758.730103:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503613312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.730149:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71282688, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x21ff0 00010000:00010000:3.0:1789760758.730159:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.730173:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.730178:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.730185:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.730198:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.734384:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71282688 -> 71286784 00002000:00010000:3.0:1789760758.734406:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139248 -> 139256 00010000:00010000:3.0:1789760758.734421:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.734433:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.734442:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.734448:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.734453:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.734462:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.734483:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.734489:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71282688->71286783] (req 71282688->71286783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.734609:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503613312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4501us (4786us total) trans 4294984710 rc 0/0 00000100:00100000:3.0:1789760758.734620:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17414 00000100:00100000:3.0:1789760758.734632:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688833441625 time 688854697217 state 0x2 ref 0 00002000:00010000:3.0:1789760758.747800:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71286784 -> 71290880 00002000:00010000:3.0:1789760758.747806:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139256 -> 139264 00010000:00010000:3.0:1789760758.747819:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ee3ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71286784->71290879] (req 71286784->71290879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.747832:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ee3ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71286784->71290879] (req 71286784->71290879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.747841:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ee3ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71286784->71290879] (req 71286784->71290879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.747849:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.747854:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ee3ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71286784->71290879] (req 71286784->71290879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.747863:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ee3ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71286784->71290879] (req 71286784->71290879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.747884:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ee3ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71286784->71290879] (req 71286784->71290879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.747891:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ee3ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71286784->71290879] (req 71286784->71290879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.747999:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503613440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4343us (6307us total) trans 4294984711 rc 0/0 00000100:00100000:3.0:1789760758.748014:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17415 00000100:00100000:3.0:1789760758.748024:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688854957633 time 688868247359 state 0x2 ref 0 00000100:00100000:3.0:1789760758.751170:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503613568 00000100:00100000:3.0:1789760758.751181:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503613568 00000100:00100000:3.0:1789760758.751237:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688854957633 time 688868247359 state 0x0 ref 1 00000100:00100000:3.0:1789760758.751252:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688868507775 time 688868247359 state 0x1 ref 1 00000100:00100000:3.0:1789760758.751285:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688868507775 time 688875891149 state 0x0 ref 1 00000100:00100000:3.0:1789760758.751297:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503613568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.751343:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71290880, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x22000 00010000:00010000:3.0:1789760758.751354:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.751370:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.751374:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.751380:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71290880->71294975] (req 71290880->71294975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.751428:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee3f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71290880->71294975] (req 71290880->71294975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.774167:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503613824 00000100:00100000:3.0:1789760758.774178:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503613824 00000100:00100000:3.0:1789760758.774589:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688876151565 time 688884987311 state 0x0 ref 1 00000100:00100000:3.0:1789760758.774605:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688885247727 time 688884987311 state 0x1 ref 1 00000100:00100000:3.0:1789760758.774633:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688885247727 time 688899239788 state 0x0 ref 1 00000100:00100000:3.0:1789760758.774656:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503613824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.774705:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71299072, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x22010 00010000:00010000:3.0:1789760758.774714:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.774729:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.774733:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.774749:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.774763:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.780545:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71299072 -> 71303168 00002000:00010000:3.0:1789760758.780550:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139280 -> 139288 00010000:00010000:3.0:1789760758.780564:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.780576:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.780590:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.780597:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.780602:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.780611:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.780631:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.780638:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee402 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71299072->71303167] (req 71299072->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.788620:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503613952 00000100:00100000:3.0:1789760758.788631:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503613952 00000100:00100000:3.0:1789760758.789365:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688885247727 time 688899239788 state 0x0 ref 1 00000100:00100000:3.0:1789760758.789384:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688899500204 time 688899239788 state 0x1 ref 1 00000100:00100000:3.0:1789760758.789839:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688899500204 time 688914443581 state 0x0 ref 1 00000100:00100000:3.0:1789760758.789860:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503613952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.789917:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71303168, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x22018 00010000:00010000:3.0:1789760758.789925:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.789941:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.789946:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.789952:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.789966:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.796514:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71303168 -> 71307264 00002000:00010000:3.0:1789760758.796519:0:8434:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760757 -> 1789760758 00002000:00010000:3.0:1789760758.796522:0:8434:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760757 -> 1789760758 00002000:00010000:3.0:1789760758.796524:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139288 -> 139296 00010000:00010000:3.0:1789760758.796538:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.796550:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.796559:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.796567:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.796573:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.796582:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.796601:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.796609:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee409 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71303168->71307263] (req 71303168->71307263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.797050:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503613952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7192us (8535us total) trans 4294984715 rc 0/0 00000100:00100000:3.0:1789760758.797060:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17419 00000100:00100000:3.0:1789760758.797070:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688899500204 time 688914443581 state 0x2 ref 0 00000001:00080000:3.0:1789760758.820984:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984717 is committed 00000001:00080000:3.0:1789760758.841660:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984718 is committed 00000100:00100000:3.0:1789760758.847192:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503614464 00000100:00100000:3.0:1789760758.847202:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503614464 00000100:00100000:3.0:1789760758.847257:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688939988885 time 688959833784 state 0x0 ref 1 00000100:00100000:3.0:1789760758.847272:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688960094200 time 688959833784 state 0x1 ref 1 00000100:00100000:3.0:1789760758.847304:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688960094200 time 688971910769 state 0x0 ref 1 00000100:00100000:3.0:1789760758.847316:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503614464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.847358:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71319552, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22038 00010000:00010000:3.0:1789760758.847368:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.847382:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760758.847387:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.847432:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.847446:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.862167:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71319552 -> 71323648 00002000:00010000:3.0:1789760758.862175:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139320 -> 139328 00010000:00010000:3.0:1789760758.862189:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.862205:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.862215:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.862223:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.862228:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.862241:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.862260:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.862268:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee425 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71319552->71323647] (req 71319552->71323647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.862377:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503614464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15058us (15464us total) trans 4294984719 rc 0/0 00000100:00100000:3.0:1789760758.862386:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17423 00000100:00100000:3.0:1789760758.862427:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 688960094200 time 688971910769 state 0x2 ref 0 00000001:00080000:3.0:1789760758.899253:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984722 is committed 00000100:00100000:3.0:1789760758.902789:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503614976 00000100:00100000:3.0:1789760758.902799:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503614976 00000100:00100000:3.0:1789760758.902861:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689008242803 time 689016974676 state 0x0 ref 1 00000100:00100000:3.0:1789760758.902876:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689017235092 time 689016974676 state 0x1 ref 1 00000100:00100000:3.0:1789760758.902938:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689017235092 time 689027544309 state 0x0 ref 1 00000100:00100000:3.0:1789760758.902952:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503614976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.902995:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71335936, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22058 00010000:00010000:3.0:1789760758.903006:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.903021:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760758.903026:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.903031:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.903046:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.907711:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71335936 -> 71340032 00002000:00010000:3.0:1789760758.907715:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139352 -> 139360 00010000:00010000:3.0:1789760758.907728:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.907754:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.907764:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.907771:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.907837:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.907849:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.907871:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.907879:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee441 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71335936->71340031] (req 71335936->71340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.907969:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503614976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5014us (5288us total) trans 4294984723 rc 0/0 00000100:00100000:3.0:1789760758.907979:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17427 00000100:00100000:3.0:1789760758.907989:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689017235092 time 689027544309 state 0x2 ref 0 00000100:00100000:3.0:1789760758.933638:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503615232 00000100:00100000:3.0:1789760758.933649:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503615232 00000100:00100000:3.0:1789760758.933706:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689027804725 time 689040775246 state 0x0 ref 1 00000100:00100000:3.0:1789760758.933721:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689041035662 time 689040775246 state 0x1 ref 1 00000100:00100000:3.0:1789760758.933764:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689041035662 time 689058370442 state 0x0 ref 1 00000100:00100000:3.0:1789760758.933785:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503615232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.933829:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71344128, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22068 00010000:00010000:3.0:1789760758.933837:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.933850:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760758.933854:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.933860:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.933873:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.938057:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71344128 -> 71348224 00002000:00010000:3.0:1789760758.938062:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139368 -> 139376 00010000:00010000:3.0:1789760758.938076:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.938088:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.938097:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.938104:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.938109:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.938118:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.938138:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.938152:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee44f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71344128->71348223] (req 71344128->71348223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.938281:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503615232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4501us (5421us total) trans 4294984725 rc 0/0 00000100:00100000:3.0:1789760758.938291:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17429 00000100:00100000:3.0:1789760758.938301:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689041035662 time 689058370442 state 0x2 ref 0 00000100:00100000:3.0:1789760758.951566:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503615488 00000100:00100000:3.0:1789760758.951575:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503615488 00000100:00100000:3.0:1789760758.951627:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689058630858 time 689068471964 state 0x0 ref 1 00000100:00100000:3.0:1789760758.951641:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689068732380 time 689068471964 state 0x1 ref 1 00000100:00100000:3.0:1789760758.951672:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689068732380 time 689076278458 state 0x0 ref 1 00000100:00100000:3.0:1789760758.951684:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503615488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760758.951730:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71352320, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22078 00010000:00010000:3.0:1789760758.951748:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.951764:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760758.951769:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.951775:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.951794:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.957105:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71352320 -> 71356416 00002000:00010000:3.0:1789760758.957111:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139384 -> 139392 00010000:00010000:3.0:1789760758.957124:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.957136:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.957145:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.957153:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.957158:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.957167:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.957186:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.957194:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee45d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71352320->71356415] (req 71352320->71356415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.957285:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503615488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5597us (5893us total) trans 4294984727 rc 0/0 00000100:00100000:3.0:1789760758.957294:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17431 00000100:00100000:3.0:1789760758.957304:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689068732380 time 689076278458 state 0x2 ref 0 00002000:00010000:3.0:1789760758.975897:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71356416 -> 71360512 00002000:00010000:3.0:1789760758.975903:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139392 -> 139400 00010000:00010000:3.0:1789760758.975917:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ee464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71356416->71360511] (req 71356416->71360511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.975929:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ee464 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71356416->71360511] (req 71356416->71360511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.975938:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ee464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71356416->71360511] (req 71356416->71360511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.975946:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760758.975952:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ee464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71356416->71360511] (req 71356416->71360511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.975962:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ee464 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71356416->71360511] (req 71356416->71360511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.975982:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ee464 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71356416->71360511] (req 71356416->71360511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.975995:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ee464 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71356416->71360511] (req 71356416->71360511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.976138:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503615616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11392us (11879us total) trans 4294984728 rc 0/0 00000100:00100000:3.0:1789760758.976150:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17432 00000100:00100000:3.0:1789760758.976162:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689076538874 time 689089333569 state 0x2 ref 0 00000100:00100000:3.0:1789760759.002769:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.011000:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503616128 00000100:00100000:3.0:1789760759.011011:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503616128 00000100:00100000:3.0:1789760759.011073:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689119484462 time 689127670347 state 0x0 ref 1 00000100:00100000:3.0:1789760759.011088:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689127930763 time 689127670347 state 0x1 ref 1 00000100:00100000:3.0:1789760759.011120:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689127930763 time 689135726838 state 0x0 ref 1 00000100:00100000:3.0:1789760759.011133:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503616128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.011178:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71372800, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x220a0 00010000:00010000:3.0:1789760759.011188:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee480 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.011203:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.011207:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.011214:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71372800->71376895] (req 71372800->71376895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.011228:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71372800->71376895] (req 71372800->71376895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.019240:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984732 is committed 00000100:00100000:3.0:1789760759.033381:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503616256 00000100:00100000:3.0:1789760759.033407:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503616256 00000100:00100000:3.0:1789760759.033524:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689127930763 time 689135726838 state 0x0 ref 1 00000100:00100000:3.0:1789760759.033541:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689135987254 time 689135726838 state 0x1 ref 1 00000100:00100000:3.0:1789760759.033612:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689135987254 time 689158217817 state 0x0 ref 1 00000100:00100000:3.0:1789760759.033625:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6aba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503616256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.033674:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71376896, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x220a8 00010000:00010000:3.0:1789760759.033683:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.033697:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.033701:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.033706:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.033719:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.043933:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71376896 -> 71380992 00002000:00010000:3.0:1789760759.043942:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139432 -> 139440 00010000:00010000:3.0:1789760759.043955:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.043969:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.043977:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.043985:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760759.043989:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.043999:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.044018:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.044025:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee487 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71376896->71380991] (req 71376896->71380991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.044249:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6aba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503616256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10621us (10998us total) trans 4294984733 rc 0/0 00000100:00100000:3.0:1789760759.044261:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17437 00000100:00100000:3.0:1789760759.044270:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689135987254 time 689158217817 state 0x2 ref 0 00000001:00080000:3.0:1789760759.095249:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984736 is committed 00000100:00100000:3.0:1789760759.108723:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503616768 00000100:00100000:3.0:1789760759.108748:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503616768 00000100:00100000:3.0:1789760759.108813:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689199948013 time 689211773166 state 0x0 ref 1 00000100:00100000:3.0:1789760759.108829:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689212033582 time 689211773166 state 0x1 ref 1 00000100:00100000:3.0:1789760759.108862:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689212033582 time 689233468103 state 0x0 ref 1 00000100:00100000:3.0:1789760759.108877:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503616768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.108922:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71393280, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x220c8 00010000:00010000:3.0:1789760759.108932:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee4a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.108946:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.108951:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.108957:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee4a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71393280->71397375] (req 71393280->71397375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.108970:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee4a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71393280->71397375] (req 71393280->71397375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.121248:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984737 is committed 00000100:00100000:3.0:1789760759.125755:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760759.132371:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984738 is committed 00000001:00080000:3.0:1789760759.172582:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984740 is committed 00000100:00100000:3.0:1789760759.179631:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503617280 00000100:00100000:3.0:1789760759.179643:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503617280 00000100:00100000:3.0:1789760759.179700:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689272844827 time 689291740492 state 0x0 ref 1 00000100:00100000:3.0:1789760759.179715:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689292000908 time 689291740492 state 0x1 ref 1 00000100:00100000:3.0:1789760759.179758:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689292000908 time 689304364247 state 0x0 ref 1 00000100:00100000:3.0:1789760759.179770:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503617280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.179819:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71409664, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x220e8 00010000:00010000:3.0:1789760759.179829:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.179845:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.179850:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.179856:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.179871:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.185859:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71409664 -> 71413760 00002000:00010000:3.0:1789760759.185865:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139496 -> 139504 00010000:00010000:3.0:1789760759.185876:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.185889:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.185897:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.185904:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760759.185908:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.185917:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.185936:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.185944:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71409664->71413759] (req 71409664->71413759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.186041:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503617280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6267us (6500us total) trans 4294984741 rc 0/0 00000100:00100000:3.0:1789760759.186051:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17445 00000100:00100000:3.0:1789760759.186060:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689292000908 time 689304364247 state 0x2 ref 0 00000100:00100000:3.0:1789760759.216065:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503617664 00000100:00100000:3.0:1789760759.216076:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503617664 00000100:00100000:3.0:1789760759.216137:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689314722742 time 689330523223 state 0x0 ref 1 00000100:00100000:3.0:1789760759.216153:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689330783639 time 689330523223 state 0x1 ref 1 00000100:00100000:3.0:1789760759.216189:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689330783639 time 689340794826 state 0x0 ref 1 00000100:00100000:3.0:1789760759.216201:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503617664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.216249:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71421952, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22100 00010000:00010000:3.0:1789760759.216257:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.216273:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.216277:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.216284:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71421952->71426047] (req 71421952->71426047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.216297:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee4d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71421952->71426047] (req 71421952->71426047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.229565:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503617792 00000100:00100000:3.0:1789760759.229578:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503617792 00000100:00100000:3.0:1789760759.296045:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689381313899 time 689404870597 state 0x0 ref 1 00000100:00100000:3.0:1789760759.296061:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689405131013 time 689404870597 state 0x1 ref 1 00000100:00100000:3.0:1789760759.296100:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689405131013 time 689420706434 state 0x0 ref 1 00000100:00100000:3.0:1789760759.296117:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503618304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.296164:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71442432, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22128 00010000:00010000:3.0:1789760759.296173:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee4f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.296191:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.296196:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.296202:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee4f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71442432->71446527] (req 71442432->71446527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.296215:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee4f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71442432->71446527] (req 71442432->71446527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.311194:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984749 is committed 00010000:00010000:3.0:1789760759.316243:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760759.424647:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984755 is committed 00000001:00080000:3.0:1789760759.450763:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984756 is committed 00000001:00080000:3.0:1789760759.461728:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984757 is committed 00000001:00080000:3.0:1789760759.481326:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984758 is committed 00000001:00080000:3.0:1789760759.508358:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984760 is committed 00000100:00100000:3.0:1789760759.511962:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503619840 00000100:00100000:3.0:1789760759.511972:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503619840 00000100:00100000:3.0:1789760759.512027:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689617950457 time 689627496690 state 0x0 ref 1 00000100:00100000:3.0:1789760759.512041:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689627757106 time 689627496690 state 0x1 ref 1 00000100:00100000:3.0:1789760759.512070:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689627757106 time 689636676236 state 0x0 ref 1 00000100:00100000:3.0:1789760759.512160:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503619840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.512205:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71491584, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22188 00010000:00010000:3.0:1789760759.512215:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee54b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.512230:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.512234:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.512240:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee54b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71491584->71495679] (req 71491584->71495679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.512252:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee54b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71491584->71495679] (req 71491584->71495679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.519170:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984761 is committed 00000001:00080000:3.0:1789760759.541281:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984763 is committed 00000001:00080000:3.0:1789760759.564515:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984764 is committed 00000100:00100000:3.0:1789760759.574090:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760759.580620:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984765 is committed 00000100:00100000:3.0:1789760759.589653:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.619939:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.630493:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.639133:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.650125:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.679847:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760759.689112:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984774 is committed 00000100:00100000:3.0:1789760759.692886:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760759.698132:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984775 is committed 00000001:00080000:3.0:1789760759.708124:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984776 is committed 00000100:00100000:3.0:1789760759.711587:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.721916:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760759.726939:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984778 is committed 00000001:00080000:3.0:1789760759.742255:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984779 is committed 00000100:00100000:3.0:1789760759.749895:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760759.759525:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503622400 00000100:00100000:3.0:1789760759.759535:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503622400 00000100:00100000:3.0:1789760759.759599:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689858987699 time 689874837533 state 0x0 ref 1 00000100:00100000:3.0:1789760759.759615:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689875097949 time 689874837533 state 0x1 ref 1 00000100:00100000:3.0:1789760759.759648:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689875097949 time 689884253807 state 0x0 ref 1 00000100:00100000:3.0:1789760759.759661:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503622400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.759710:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71573504, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x22228 00010000:00010000:3.0:1789760759.759720:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee5d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.759750:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.759755:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.759762:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee5d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71573504->71577599] (req 71573504->71577599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.759785:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee5d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71573504->71577599] (req 71573504->71577599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.773643:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984782 is committed 00000100:00100000:3.0:1789760759.811080:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503622912 00000100:00100000:3.0:1789760759.811094:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503622912 00000100:00100000:3.0:1789760759.811153:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689909780173 time 689924632631 state 0x0 ref 1 00000100:00100000:3.0:1789760759.811170:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689924893047 time 689924632631 state 0x1 ref 1 00000100:00100000:3.0:1789760759.811244:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689924893047 time 689935849364 state 0x0 ref 1 00000100:00100000:3.0:1789760759.811258:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503622912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.811308:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71589888, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x22248 00010000:00010000:3.0:1789760759.811318:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.811335:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.811340:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.811347:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.811363:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.815149:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71589888 -> 71593984 00002000:00010000:3.0:1789760759.815155:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139848 -> 139856 00010000:00010000:3.0:1789760759.815168:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.815180:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.815190:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.815197:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760759.815201:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.815210:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.815230:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.815238:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee5f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71589888->71593983] (req 71589888->71593983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.815348:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503622912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4088us (5466us total) trans 4294984785 rc 0/0 00000100:00100000:3.0:1789760759.815357:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17489 00000100:00100000:3.0:1789760759.815366:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689924893047 time 689935849364 state 0x2 ref 0 00000001:00080000:3.0:1789760759.828845:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984786 is committed 00000001:00080000:3.0:1789760759.840139:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984787 is committed 00000001:00080000:3.0:1789760759.849347:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984788 is committed 00000100:00100000:3.0:1789760759.857608:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503623424 00000100:00100000:3.0:1789760759.857618:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503623424 00000100:00100000:3.0:1789760759.857672:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689957857921 time 689969143462 state 0x0 ref 1 00000100:00100000:3.0:1789760759.857687:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689969403878 time 689969143462 state 0x1 ref 1 00000100:00100000:3.0:1789760759.857718:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 689969403878 time 689982325030 state 0x0 ref 1 00000100:00100000:3.0:1789760759.857740:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503623424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760759.857903:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71606272, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x22268 00010000:00010000:3.0:1789760759.857913:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee60f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.857929:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760759.857934:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.857941:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee60f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71606272->71610367] (req 71606272->71610367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.857959:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee60f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71606272->71610367] (req 71606272->71610367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.874046:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984790 is committed 00000001:00080000:3.0:1789760759.894995:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984792 is committed 00000001:00080000:3.0:1789760759.906840:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984793 is committed 00000001:00080000:3.0:1789760759.930433:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984795 is committed 00000001:00080000:3.0:1789760759.941770:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984796 is committed 00002000:00010000:3.0:1789760759.958767:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71639040 -> 71643136 00002000:00010000:3.0:1789760759.958772:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139944 -> 139952 00010000:00010000:3.0:1789760759.958785:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ee647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71639040->71643135] (req 71639040->71643135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.958796:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ee647 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71639040->71643135] (req 71639040->71643135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.958804:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ee647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71639040->71643135] (req 71639040->71643135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.958810:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760759.958814:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ee647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71639040->71643135] (req 71639040->71643135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.958822:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ee647 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71639040->71643135] (req 71639040->71643135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.958839:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ee647 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71639040->71643135] (req 71639040->71643135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.958845:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ee647 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71639040->71643135] (req 71639040->71643135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.958975:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503624448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8685us (8913us total) trans 4294984797 rc 0/0 00000100:00100000:3.0:1789760759.958985:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17501 00000100:00100000:3.0:1789760759.958996:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690057991372 time 690074879171 state 0x2 ref 0 00000100:00100000:3.0:1789760760.019686:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503624960 00000100:00100000:3.0:1789760760.019691:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503624960 00000100:00100000:3.0:1789760760.019745:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690117030227 time 690128844874 state 0x0 ref 1 00000100:00100000:3.0:1789760760.019761:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690129105290 time 690128844874 state 0x1 ref 1 00000100:00100000:3.0:1789760760.019787:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690129105290 time 690144393293 state 0x0 ref 1 00000100:00100000:3.0:1789760760.019797:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503624960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.019837:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71655424, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x222c8 00010000:00010000:3.0:1789760760.019846:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.019861:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760760.019866:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.019871:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.019884:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.031150:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71655424 -> 71659520 00002000:00010000:3.0:1789760760.031161:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 139976 -> 139984 00010000:00010000:3.0:1789760760.031174:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.031192:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.031200:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.031208:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760760.031212:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.031221:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.031240:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.031247:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee663 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71655424->71659519] (req 71655424->71659519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.031349:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503624960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11548us (11722us total) trans 4294984801 rc 0/0 00000100:00100000:3.0:1789760760.031360:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17505 00000100:00100000:3.0:1789760760.031369:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690129105290 time 690144393293 state 0x2 ref 0 00000100:00100000:3.0:1789760760.076840:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503625472 00000100:00100000:3.0:1789760760.076850:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503625472 00000100:00100000:3.0:1789760760.076902:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690171924832 time 690183799601 state 0x0 ref 1 00000100:00100000:3.0:1789760760.076916:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690184060017 time 690183799601 state 0x1 ref 1 00000100:00100000:3.0:1789760760.076949:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690184060017 time 690201556141 state 0x0 ref 1 00000100:00100000:3.0:1789760760.076963:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503625472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.077009:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71671808, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x222e8 00010000:00010000:3.0:1789760760.077019:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.077034:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760760.077039:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.077045:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.077057:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.084891:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71671808 -> 71675904 00002000:00010000:3.0:1789760760.084897:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140008 -> 140016 00010000:00010000:3.0:1789760760.084912:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.084925:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.084939:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.084947:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760760.084952:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.084961:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.084980:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.084988:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee67f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71671808->71675903] (req 71671808->71675903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.085097:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503625472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8132us (8360us total) trans 4294984805 rc 0/0 00000100:00100000:3.0:1789760760.085109:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17509 00000100:00100000:3.0:1789760760.085120:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690184060017 time 690201556141 state 0x2 ref 0 00000100:00100000:3.0:1789760760.125078:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503625984 00000100:00100000:3.0:1789760760.125089:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503625984 00000100:00100000:3.0:1789760760.125141:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690229209364 time 690238639571 state 0x0 ref 1 00000100:00100000:3.0:1789760760.125157:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690238899987 time 690238639571 state 0x1 ref 1 00000100:00100000:3.0:1789760760.125190:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690238899987 time 690249796232 state 0x0 ref 1 00000100:00100000:3.0:1789760760.125202:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503625984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.125245:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71688192, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x22308 00010000:00010000:3.0:1789760760.125254:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.125269:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760760.125273:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.125278:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.125291:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.135122:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71688192 -> 71692288 00002000:00010000:3.0:1789760760.135129:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140040 -> 140048 00010000:00010000:3.0:1789760760.135143:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.135157:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.135165:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.135178:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760760.135183:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.135193:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.135213:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.135220:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee69b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71688192->71692287] (req 71688192->71692287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.135341:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503625984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10136us (10351us total) trans 4294984809 rc 0/0 00000100:00100000:3.0:1789760760.135351:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17513 00000100:00100000:3.0:1789760760.135359:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690238899987 time 690249796232 state 0x2 ref 0 00000100:00100000:3.0:1789760760.182257:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503626496 00000100:00100000:3.0:1789760760.182267:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503626496 00000100:00100000:3.0:1789760760.182328:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690279574810 time 690293148644 state 0x0 ref 1 00000100:00100000:3.0:1789760760.182343:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690293409060 time 690293148644 state 0x1 ref 1 00000100:00100000:3.0:1789760760.182482:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690293409060 time 690307087280 state 0x0 ref 1 00000100:00100000:3.0:1789760760.182497:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503626496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.182539:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71704576, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x22328 00010000:00010000:3.0:1789760760.182547:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee6b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.182562:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760760.182566:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.182572:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee6b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71704576->71708671] (req 71704576->71708671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.182584:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ee6b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71704576->71708671] (req 71704576->71708671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.268558:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984817 is committed 00000001:00080000:3.0:1789760760.280809:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984818 is committed 00000001:00080000:3.0:1789760760.294968:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984819 is committed 00000001:00080000:3.0:1789760760.340672:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984822 is committed 00000001:00080000:3.0:1789760760.361346:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984823 is committed 00000100:00100000:3.0:1789760760.624216:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760760.637937:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71819264 -> 71823360 00002000:00010000:3.0:1789760760.637943:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140296 -> 140304 00010000:00010000:3.0:1789760760.637957:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ee77b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71819264->71823359] (req 71819264->71823359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.637969:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ee77b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71819264->71823359] (req 71819264->71823359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.637983:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ee77b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71819264->71823359] (req 71819264->71823359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.637990:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760760.637996:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ee77b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71819264->71823359] (req 71819264->71823359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.638004:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ee77b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71819264->71823359] (req 71819264->71823359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.638025:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ee77b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71819264->71823359] (req 71819264->71823359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.638032:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083ee77b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71819264->71823359] (req 71819264->71823359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.638162:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503630080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13275us (13949us total) trans 4294984841 rc 0/0 00000100:00100000:3.0:1789760760.638174:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17545 00000100:00100000:3.0:1789760760.638183:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690735878110 time 690749478279 state 0x2 ref 0 00000100:00100000:3.0:1789760760.685498:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760760.702192:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503630592 00000100:00100000:3.0:1789760760.702203:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503630592 00000100:00100000:3.0:1789760760.702258:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690788357770 time 690810581760 state 0x0 ref 1 00000100:00100000:3.0:1789760760.702277:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690810842176 time 690810581760 state 0x1 ref 1 00000100:00100000:3.0:1789760760.702308:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690810842176 time 690826915211 state 0x0 ref 1 00000100:00100000:3.0:1789760760.702321:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503630592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.702370:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71835648, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x22428 00010000:00010000:3.0:1789760760.702379:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.702429:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760760.702435:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.702441:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.702564:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.712383:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71835648 -> 71839744 00002000:00010000:3.0:1789760760.712432:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140328 -> 140336 00010000:00010000:3.0:1789760760.712451:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.712464:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.712472:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.712479:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760760.712484:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.712492:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.712510:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.712518:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee797 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71835648->71839743] (req 71835648->71839743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.712687:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503630592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10363us (10588us total) trans 4294984845 rc 0/0 00000100:00100000:3.0:1789760760.712698:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17549 00000100:00100000:3.0:1789760760.712707:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690810842176 time 690826915211 state 0x2 ref 0 00000100:00100000:3.0:1789760760.759927:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503631104 00000100:00100000:3.0:1789760760.759937:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503631104 00000100:00100000:3.0:1789760760.759987:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690861702232 time 690871664916 state 0x0 ref 1 00000100:00100000:3.0:1789760760.760001:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690871925332 time 690871664916 state 0x1 ref 1 00000100:00100000:3.0:1789760760.760033:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690871925332 time 690884639739 state 0x0 ref 1 00000100:00100000:3.0:1789760760.760048:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503631104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.760094:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71852032, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x22448 00010000:00010000:3.0:1789760760.760103:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.760118:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760760.760122:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.760129:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.760143:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.772125:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71852032 -> 71856128 00002000:00010000:3.0:1789760760.772131:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140360 -> 140368 00010000:00010000:3.0:1789760760.772143:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.772155:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.772164:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.772172:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760760.772177:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.772187:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.772205:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.772212:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71852032->71856127] (req 71852032->71856127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.772304:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503631104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12253us (12477us total) trans 4294984849 rc 0/0 00000100:00100000:3.0:1789760760.772314:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17553 00000100:00100000:3.0:1789760760.772323:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690871925332 time 690884639739 state 0x2 ref 0 00000100:00100000:3.0:1789760760.778121:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760760.829285:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760760.886787:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760760.904160:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503632128 00000100:00100000:3.0:1789760760.904170:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503632128 00000100:00100000:3.0:1789760760.904222:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 690992936067 time 691011809449 state 0x0 ref 1 00000100:00100000:3.0:1789760760.904236:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691012069865 time 691011809449 state 0x1 ref 1 00000100:00100000:3.0:1789760760.904311:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691012069865 time 691028917211 state 0x0 ref 1 00000100:00100000:3.0:1789760760.904327:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503632128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.904370:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71884800, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x22488 00010000:00010000:3.0:1789760760.904379:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.904430:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760760.904435:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.904441:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71884800->71888895] (req 71884800->71888895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.904467:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ee7eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71884800->71888895] (req 71884800->71888895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.911736:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984857 is committed 00000001:00080000:3.0:1789760760.924905:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984858 is committed 00000100:00100000:3.0:1789760760.934628:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760760.941671:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984859 is committed 00000100:00100000:3.0:1789760760.970570:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503632640 00000100:00100000:3.0:1789760760.970580:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503632640 00000100:00100000:3.0:1789760760.970635:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691059781228 time 691084641675 state 0x0 ref 1 00000100:00100000:3.0:1789760760.970649:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691084902091 time 691084641675 state 0x1 ref 1 00000100:00100000:3.0:1789760760.970680:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691084902091 time 691095286713 state 0x0 ref 1 00000100:00100000:3.0:1789760760.970692:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff519450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503632640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760760.970748:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71901184, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x224a8 00010000:00010000:3.0:1789760760.970757:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.970782:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760760.970787:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.970793:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.970807:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.979031:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71901184 -> 71905280 00002000:00010000:3.0:1789760760.979036:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140456 -> 140464 00010000:00010000:3.0:1789760760.979048:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979059:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979080:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979088:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760760.979092:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979106:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979138:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.979145:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee807 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71901184->71905279] (req 71901184->71905279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.979317:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff519450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503632640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8622us (8958us total) trans 4294984861 rc 0/0 00000100:00100000:3.0:1789760760.979328:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17565 00000100:00100000:3.0:1789760760.979338:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691084902091 time 691095286713 state 0x2 ref 0 00000100:00100000:3.0:1789760760.983049:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760761.003373:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984863 is committed 00000001:00080000:3.0:1789760761.018438:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984864 is committed 00000100:00100000:3.0:1789760761.022933:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760761.028000:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984865 is committed 00000001:00080000:3.0:1789760761.042115:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984866 is committed 00000001:00080000:3.0:1789760761.053253:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984867 is committed 00000100:00100000:3.0:1789760761.056774:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760761.064193:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984868 is committed 00000100:00100000:3.0:1789760761.067663:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503633664 00000100:00100000:3.0:1789760761.067673:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503633664 00000100:00100000:3.0:1789760761.067733:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691172984825 time 691181590242 state 0x0 ref 1 00000100:00100000:3.0:1789760761.067750:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691181850658 time 691181590242 state 0x1 ref 1 00000100:00100000:3.0:1789760761.067790:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691181850658 time 691192397244 state 0x0 ref 1 00000100:00100000:3.0:1789760761.067803:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503633664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.067849:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71933952, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x224e8 00010000:00010000:3.0:1789760761.067859:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee83f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.067875:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.067879:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.067886:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee83f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71933952->71938047] (req 71933952->71938047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.067900:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ee83f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71933952->71938047] (req 71933952->71938047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.073054:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984869 is committed 00000001:00080000:3.0:1789760761.088019:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984870 is committed 00000100:00100000:3.0:1789760761.094251:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760761.100740:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984871 is committed 00000100:00100000:3.0:1789760761.131749:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503634176 00000100:00100000:3.0:1789760761.131769:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503634176 00000100:00100000:3.0:1789760761.131825:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691219363504 time 691244112358 state 0x0 ref 1 00000100:00100000:3.0:1789760761.131840:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691244372774 time 691244112358 state 0x1 ref 1 00000100:00100000:3.0:1789760761.131903:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691244372774 time 691256509379 state 0x0 ref 1 00000100:00100000:3.0:1789760761.131942:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503634176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.131989:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71950336, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x22508 00010000:00010000:3.0:1789760761.131998:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.132014:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.132018:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.132024:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.132038:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.139216:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71950336 -> 71954432 00002000:00010000:3.0:1789760761.139221:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140552 -> 140560 00010000:00010000:3.0:1789760761.139234:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.139247:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.139256:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.139268:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760761.139272:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.139281:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.139302:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.139310:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee85b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71950336->71954431] (req 71950336->71954431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.139434:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503634176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7489us (7784us total) trans 4294984873 rc 0/0 00000100:00100000:3.0:1789760761.139445:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17577 00000100:00100000:3.0:1789760761.139456:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691244372774 time 691256509379 state 0x2 ref 0 00000100:00100000:3.0:1789760761.146148:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760761.152669:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984874 is committed 00000001:00080000:3.0:1789760761.166133:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984875 is committed 00000100:00100000:3.0:1789760761.181891:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760761.186229:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984877 is committed 00000001:00080000:3.0:1789760761.199142:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984878 is committed 00000001:00080000:3.0:1789760761.214784:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984879 is committed 00000100:00100000:3.0:1789760761.218224:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760761.222549:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984880 is committed 00000100:00100000:3.0:1789760761.226143:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503635200 00000100:00100000:3.0:1789760761.226153:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503635200 00000100:00100000:3.0:1789760761.226209:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691330135410 time 691343051530 state 0x0 ref 1 00000100:00100000:3.0:1789760761.226223:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691343311946 time 691343051530 state 0x1 ref 1 00000100:00100000:3.0:1789760761.226288:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691343311946 time 691350894608 state 0x0 ref 1 00000100:00100000:3.0:1789760761.226301:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503635200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.226350:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71983104, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x22548 00010000:00010000:3.0:1789760761.226359:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.226374:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.226378:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.226384:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.226432:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.230327:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 71983104 -> 71987200 00002000:00010000:3.0:1789760761.230333:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140616 -> 140624 00010000:00010000:3.0:1789760761.230345:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.230364:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.230373:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.230380:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760761.230384:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.230426:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.230455:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.230464:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee893 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71983104->71987199] (req 71983104->71987199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.230557:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503635200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4255us (4529us total) trans 4294984881 rc 0/0 00000100:00100000:3.0:1789760761.230568:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17585 00000100:00100000:3.0:1789760761.230578:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691343311946 time 691350894608 state 0x2 ref 0 00000001:00080000:3.0:1789760761.238890:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984882 is committed 00000100:00100000:3.0:1789760761.271145:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503635712 00000100:00100000:3.0:1789760761.271156:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503635712 00000100:00100000:3.0:1789760761.271210:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691372339698 time 691387214833 state 0x0 ref 1 00000100:00100000:3.0:1789760761.271224:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691387475249 time 691387214833 state 0x1 ref 1 00000100:00100000:3.0:1789760761.271254:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691387475249 time 691395860783 state 0x0 ref 1 00000100:00100000:3.0:1789760761.271266:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503635712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.271311:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 71999488, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x22568 00010000:00010000:3.0:1789760761.271319:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee8af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.271336:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.271341:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.271347:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee8af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71999488->72003583] (req 71999488->72003583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.271360:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ee8af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [71999488->72003583] (req 71999488->72003583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.349616:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984891 is committed 00000100:00100000:3.0:1789760761.356435:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00010000:00010000:3.0:1789760761.362531:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760761.462135:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984900 is committed 00000001:00080000:3.0:1789760761.473572:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984901 is committed 00000001:00080000:3.0:1789760761.482279:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984902 is committed 00000001:00080000:3.0:1789760761.516383:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984905 is committed 00000100:00100000:3.0:1789760761.558955:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760761.559097:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6969180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523601408:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760761.559158:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760761.559595:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6969180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523601408:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760761.583562:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503638912 00000100:00100000:3.0:1789760761.583573:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503638912 00000100:00100000:3.0:1789760761.583637:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691682552389 time 691696501899 state 0x0 ref 1 00000100:00100000:3.0:1789760761.583653:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691696762315 time 691696501899 state 0x1 ref 1 00000100:00100000:3.0:1789760761.583683:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691696762315 time 691708289542 state 0x0 ref 1 00000100:00100000:3.0:1789760761.583694:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503638912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.583747:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72101888, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x22630 00010000:00010000:3.0:1789760761.583766:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.583782:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.583787:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.583793:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.583807:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.589514:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72101888 -> 72105984 00002000:00010000:3.0:1789760761.589519:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140848 -> 140856 00010000:00010000:3.0:1789760761.589532:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.589545:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.589554:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.589562:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760761.589567:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.589575:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.589595:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.589604:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee95e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72101888->72105983] (req 72101888->72105983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.589744:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503638912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6046us (6393us total) trans 4294984910 rc 0/0 00000100:00100000:3.0:1789760761.589766:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17614 00000100:00100000:3.0:1789760761.589776:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691696762315 time 691708289542 state 0x2 ref 0 00000001:00080000:3.0:1789760761.617558:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984911 is committed 00000100:00100000:3.0:1789760761.706246:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503639424 00000100:00100000:3.0:1789760761.706256:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503639424 00000100:00100000:3.0:1789760761.706311:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691767069237 time 691814133068 state 0x0 ref 1 00000100:00100000:3.0:1789760761.706326:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691814393484 time 691814133068 state 0x1 ref 1 00000100:00100000:3.0:1789760761.706383:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691814393484 time 691830989150 state 0x0 ref 1 00000100:00100000:3.0:1789760761.706412:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503639424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.706466:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72118272, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x22650 00010000:00010000:3.0:1789760761.706476:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.706493:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.706497:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.706503:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.706517:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.717552:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72118272 -> 72122368 00002000:00010000:3.0:1789760761.717558:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140880 -> 140888 00010000:00010000:3.0:1789760761.717571:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.717589:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.717598:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.717605:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760761.717610:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.717884:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.717905:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.717913:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee97a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72118272->72122367] (req 72118272->72122367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.718040:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503639424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11626us (12018us total) trans 4294984914 rc 0/0 00000100:00100000:3.0:1789760761.718050:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17618 00000100:00100000:3.0:1789760761.718061:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691814393484 time 691830989150 state 0x2 ref 0 00000100:00100000:3.0:1789760761.779022:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503639936 00000100:00100000:3.0:1789760761.779039:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503639936 00000100:00100000:3.0:1789760761.779105:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691865134255 time 691876885094 state 0x0 ref 1 00000100:00100000:3.0:1789760761.779122:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691877145510 time 691876885094 state 0x1 ref 1 00000100:00100000:3.0:1789760761.779209:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691877145510 time 691903815200 state 0x0 ref 1 00000100:00100000:3.0:1789760761.779223:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffb6d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503639936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.779271:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72134656, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x22670 00010000:00010000:3.0:1789760761.779281:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.779296:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.779301:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.779308:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.779321:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.790308:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72134656 -> 72138752 00002000:00010000:3.0:1789760761.790318:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140912 -> 140920 00010000:00010000:3.0:1789760761.790331:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.790342:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.790356:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.790363:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760761.790382:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.790433:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.790452:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.790459:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee996 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72134656->72138751] (req 72134656->72138751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.790547:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffb6d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503639936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11322us (13889us total) trans 4294984918 rc 0/0 00000100:00100000:3.0:1789760761.790556:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17622 00000100:00100000:3.0:1789760761.790566:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691877145510 time 691903815200 state 0x2 ref 0 00000100:00100000:3.0:1789760761.854605:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503640448 00000100:00100000:3.0:1789760761.854615:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503640448 00000100:00100000:3.0:1789760761.854668:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691944091628 time 691960162247 state 0x0 ref 1 00000100:00100000:3.0:1789760761.854684:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691960422663 time 691960162247 state 0x1 ref 1 00000100:00100000:3.0:1789760761.854728:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 691960422663 time 691979324226 state 0x0 ref 1 00000100:00100000:3.0:1789760761.854741:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503640448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.854795:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72151040, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22690 00010000:00010000:3.0:1789760761.854805:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee9b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.854821:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760761.854826:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.854832:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee9b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72151040->72155135] (req 72151040->72155135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.854847:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee9b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72151040->72155135] (req 72151040->72155135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.863899:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984922 is committed 00000001:00080000:3.0:1789760761.886567:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984923 is committed 00000001:00080000:3.0:1789760761.911931:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984924 is committed 00000001:00080000:3.0:1789760761.924618:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984925 is committed 00000100:00100000:3.0:1789760761.928291:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503640960 00000100:00100000:3.0:1789760761.928302:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503640960 00000100:00100000:3.0:1789760761.928359:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692028989144 time 692041885635 state 0x0 ref 1 00000100:00100000:3.0:1789760761.928375:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692042146051 time 692041885635 state 0x1 ref 1 00000100:00100000:3.0:1789760761.928434:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692042146051 time 692053039385 state 0x0 ref 1 00000100:00100000:3.0:1789760761.928458:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503640960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.928509:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72167424, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x226b0 00010000:00010000:3.0:1789760761.928518:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.928534:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760761.928538:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.928545:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.928558:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.934800:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72167424 -> 72171520 00002000:00010000:3.0:1789760761.934805:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 140976 -> 140984 00010000:00010000:3.0:1789760761.934818:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.934828:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.934834:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.934839:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760761.934843:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.934849:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.934864:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.934869:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72167424->72171519] (req 72167424->72171519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.934951:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503640960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6493us (6741us total) trans 4294984926 rc 0/0 00000100:00100000:3.0:1789760761.934958:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17630 00000100:00100000:3.0:1789760761.934965:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692042146051 time 692053039385 state 0x2 ref 0 00000001:00080000:3.0:1789760761.966056:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984929 is committed 00000001:00080000:3.0:1789760761.975631:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984930 is committed 00000001:00080000:3.0:1789760761.991212:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984931 is committed 00000100:00100000:3.0:1789760761.997981:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503641728 00000100:00100000:3.0:1789760761.997991:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503641728 00000100:00100000:3.0:1789760761.998047:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692096060621 time 692107961966 state 0x0 ref 1 00000100:00100000:3.0:1789760761.998063:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692108222382 time 692107961966 state 0x1 ref 1 00000100:00100000:3.0:1789760761.998094:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692108222382 time 692122700824 state 0x0 ref 1 00000100:00100000:3.0:1789760761.998109:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503641728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760761.998157:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72192000, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x226e0 00010000:00010000:3.0:1789760761.998168:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.998184:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760761.998189:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.998195:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72192000->72196095] (req 72192000->72196095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.998209:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ee9f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72192000->72196095] (req 72192000->72196095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.009771:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503641856 00000100:00100000:3.0:1789760762.009784:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503641856 00000100:00100000:3.0:1789760762.009857:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692108222382 time 692122700824 state 0x0 ref 1 00000100:00100000:3.0:1789760762.009872:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692122961240 time 692122700824 state 0x1 ref 1 00000100:00100000:3.0:1789760762.009904:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692122961240 time 692134510107 state 0x0 ref 1 00000100:00100000:3.0:1789760762.009917:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503641856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.009964:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72196096, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x226e8 00010000:00010000:3.0:1789760762.009973:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee9ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.009994:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.009998:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.010005:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee9ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72196096->72200191] (req 72196096->72200191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.010018:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ee9ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72196096->72200191] (req 72196096->72200191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.041947:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503642240 00000100:00100000:3.0:1789760762.041958:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503642240 00000100:00100000:3.0:1789760762.042011:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692145602977 time 692157482319 state 0x0 ref 1 00000100:00100000:3.0:1789760762.042026:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692157742735 time 692157482319 state 0x1 ref 1 00000100:00100000:3.0:1789760762.042057:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692157742735 time 692166662987 state 0x0 ref 1 00000100:00100000:3.0:1789760762.042069:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503642240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.042115:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72208384, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22700 00010000:00010000:3.0:1789760762.042124:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.042140:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.042145:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.042151:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.042168:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.054602:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72208384 -> 72212480 00002000:00010000:3.0:1789760762.054608:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141056 -> 141064 00010000:00010000:3.0:1789760762.054621:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.054633:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.054642:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.054648:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.054653:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.054673:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.054692:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.054699:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72208384->72212479] (req 72208384->72212479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.054900:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503642240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12827us (13097us total) trans 4294984936 rc 0/0 00000100:00100000:3.0:1789760762.054911:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17640 00000100:00100000:3.0:1789760762.054921:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692157742735 time 692166662987 state 0x2 ref 0 00000100:00100000:3.0:1789760762.059239:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503642368 00000100:00100000:3.0:1789760762.059249:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503642368 00000100:00100000:3.0:1789760762.059301:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692157742735 time 692166662987 state 0x0 ref 1 00000100:00100000:3.0:1789760762.059316:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692166923403 time 692166662987 state 0x1 ref 1 00000100:00100000:3.0:1789760762.059348:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692166923403 time 692183955472 state 0x0 ref 1 00000100:00100000:3.0:1789760762.059360:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503642368:12345-192.168.203.58@tcp:4:dd.0 00000100:00100000:3.0:1789760762.102925:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503642752 00000100:00100000:3.0:1789760762.102935:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503642752 00000100:00100000:3.0:1789760762.102996:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692203914378 time 692215499427 state 0x0 ref 1 00000100:00100000:3.0:1789760762.103012:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692215759843 time 692215499427 state 0x1 ref 1 00000100:00100000:3.0:1789760762.103083:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692215759843 time 692227688286 state 0x0 ref 1 00000100:00100000:3.0:1789760762.103096:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503642752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.103144:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72224768, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22720 00010000:00010000:3.0:1789760762.103156:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.103172:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.103176:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.103183:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.103197:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.112603:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72224768 -> 72228864 00002000:00010000:3.0:1789760762.112608:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141088 -> 141096 00010000:00010000:3.0:1789760762.112622:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.112746:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.112755:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.112763:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.112768:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.112776:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.112798:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.112806:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72224768->72228863] (req 72224768->72228863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.112973:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503642752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9874us (10227us total) trans 4294984940 rc 0/0 00000100:00100000:3.0:1789760762.112984:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17644 00000100:00100000:3.0:1789760762.112995:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692215759843 time 692227688286 state 0x2 ref 0 00000100:00100000:3.0:1789760762.166671:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503643264 00000100:00100000:3.0:1789760762.166683:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503643264 00000100:00100000:3.0:1789760762.166750:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692258880681 time 692277382576 state 0x0 ref 1 00000100:00100000:3.0:1789760762.166765:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692277642992 time 692277382576 state 0x1 ref 1 00000100:00100000:3.0:1789760762.166785:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692277642992 time 692291392111 state 0x0 ref 1 00000100:00100000:3.0:1789760762.166796:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503643264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.166839:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72241152, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22740 00010000:00010000:3.0:1789760762.166848:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.166864:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.166868:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.166874:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.166887:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.176431:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72241152 -> 72245248 00002000:00010000:3.0:1789760762.176555:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141120 -> 141128 00010000:00010000:3.0:1789760762.176570:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.176583:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.176593:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.176601:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.176605:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.176615:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.176699:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.176722:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72241152->72245247] (req 72241152->72245247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.176819:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503643264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10018us (10460us total) trans 4294984944 rc 0/0 00000100:00100000:3.0:1789760762.176829:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17648 00000100:00100000:3.0:1789760762.176839:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692277642992 time 692291392111 state 0x2 ref 0 00000001:00080000:3.0:1789760762.193061:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984945 is committed 00000001:00080000:3.0:1789760762.203792:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984946 is committed 00000100:00100000:3.0:1789760762.216493:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503643776 00000100:00100000:3.0:1789760762.216503:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503643776 00000100:00100000:3.0:1789760762.216556:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692322989981 time 692331584633 state 0x0 ref 1 00000100:00100000:3.0:1789760762.216570:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692331845049 time 692331584633 state 0x1 ref 1 00000100:00100000:3.0:1789760762.216631:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692331845049 time 692341236697 state 0x0 ref 1 00000100:00100000:3.0:1789760762.216644:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503643776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.216699:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72257536, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22760 00010000:00010000:3.0:1789760762.216708:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.216731:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.216735:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.216742:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.216761:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.229356:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72257536 -> 72261632 00002000:00010000:3.0:1789760762.229361:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141152 -> 141160 00010000:00010000:3.0:1789760762.229379:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.229409:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.229419:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.229427:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.229432:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.229858:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.229881:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.229894:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72257536->72261631] (req 72257536->72261631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.230049:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503643776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13402us (13778us total) trans 4294984948 rc 0/0 00000100:00100000:3.0:1789760762.230060:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17652 00000100:00100000:3.0:1789760762.230070:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692331845049 time 692341236697 state 0x2 ref 0 00000001:00080000:3.0:1789760762.269251:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984951 is committed 00000100:00100000:3.0:1789760762.275435:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503644288 00000100:00100000:3.0:1789760762.275446:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503644288 00000100:00100000:3.0:1789760762.275558:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692375275288 time 692386390256 state 0x0 ref 1 00000100:00100000:3.0:1789760762.275574:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692386650672 time 692386390256 state 0x1 ref 1 00000100:00100000:3.0:1789760762.275656:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692386650672 time 692400261192 state 0x0 ref 1 00000100:00100000:3.0:1789760762.275671:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503644288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.275727:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72273920, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22780 00010000:00010000:3.0:1789760762.275735:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.275752:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.275757:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.275763:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72273920->72278015] (req 72273920->72278015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.275778:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eea84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72273920->72278015] (req 72273920->72278015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.288197:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984952 is committed 00000001:00080000:3.0:1789760762.304482:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984953 is committed 00000100:00100000:3.0:1789760762.316022:0:6505:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503644672 00000100:00100000:3.0:1789760762.316027:0:6505:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503644672 00000100:00100000:3.0:1789760762.316192:0:6505:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 482 00000100:00100000:3.0:1789760762.316198:0:6505:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2390:x1876699503644672:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760762.316225:0:6505:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2390:x1876699503644672:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 28us (714us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760762.316231:0:6505:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 482 00000001:00080000:3.0:1789760762.323146:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984954 is committed 00000001:00080000:3.0:1789760762.335547:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984955 is committed 00000001:00080000:3.0:1789760762.347111:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984956 is committed 00000001:00080000:3.0:1789760762.358284:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984957 is committed 00010000:00010000:3.0:1789760762.386751:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760762.481789:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760762.532017:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760762.551969:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503647104 00000100:00100000:3.0:1789760762.551980:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503647104 00000100:00100000:3.0:1789760762.552035:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692644182614 time 692658082039 state 0x0 ref 1 00000100:00100000:3.0:1789760762.552051:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692658342455 time 692658082039 state 0x1 ref 1 00000100:00100000:3.0:1789760762.552083:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692658342455 time 692676689899 state 0x0 ref 1 00000100:00100000:3.0:1789760762.552096:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503647104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.552141:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72343552, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22808 00010000:00010000:3.0:1789760762.552150:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.552167:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.552171:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.552178:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.552192:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.560577:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72343552 -> 72347648 00002000:00010000:3.0:1789760762.560583:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141320 -> 141328 00010000:00010000:3.0:1789760762.560596:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.560608:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.560617:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.560624:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.560629:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.560638:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.560658:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.560666:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeafb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72343552->72347647] (req 72343552->72347647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.560841:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503647104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8742us (8967us total) trans 4294984969 rc 0/0 00000100:00100000:3.0:1789760762.560851:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17673 00000100:00100000:3.0:1789760762.560861:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692658342455 time 692676689899 state 0x2 ref 0 00000001:00080000:3.0:1789760762.613969:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984972 is committed 00000100:00100000:3.0:1789760762.618917:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503647616 00000100:00100000:3.0:1789760762.618926:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503647616 00000100:00100000:3.0:1789760762.618979:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692709131489 time 692726202950 state 0x0 ref 1 00000100:00100000:3.0:1789760762.618993:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692726463366 time 692726202950 state 0x1 ref 1 00000100:00100000:3.0:1789760762.619025:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692726463366 time 692743631865 state 0x0 ref 1 00000100:00100000:3.0:1789760762.619037:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503647616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.619083:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72359936, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22828 00010000:00010000:3.0:1789760762.619092:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.619107:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.619111:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.619118:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.619131:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.624262:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72359936 -> 72364032 00002000:00010000:3.0:1789760762.624268:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141352 -> 141360 00010000:00010000:3.0:1789760762.624282:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.624295:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.624303:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.624311:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.624316:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.624330:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.624350:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.624358:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72359936->72364031] (req 72359936->72364031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.624599:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503647616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5559us (5864us total) trans 4294984973 rc 0/0 00000100:00100000:3.0:1789760762.624610:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17677 00000100:00100000:3.0:1789760762.624619:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692726463366 time 692743631865 state 0x2 ref 0 00000100:00100000:3.0:1789760762.680092:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503648128 00000100:00100000:3.0:1789760762.680102:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503648128 00000100:00100000:3.0:1789760762.680158:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692781767416 time 692790313997 state 0x0 ref 1 00000100:00100000:3.0:1789760762.680174:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692790574413 time 692790313997 state 0x1 ref 1 00000100:00100000:3.0:1789760762.680206:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692790574413 time 692804813661 state 0x0 ref 1 00000100:00100000:3.0:1789760762.680218:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503648128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.680263:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72376320, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22848 00010000:00010000:3.0:1789760762.680272:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.680287:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.680292:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.680298:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.680312:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.686629:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72376320 -> 72380416 00002000:00010000:3.0:1789760762.686634:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141384 -> 141392 00010000:00010000:3.0:1789760762.686647:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.686659:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.686667:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.686674:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.686678:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.686686:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.686723:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.686731:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72376320->72380415] (req 72376320->72380415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.687097:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503648128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6874us (7194us total) trans 4294984977 rc 0/0 00000100:00100000:3.0:1789760762.687109:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17681 00000100:00100000:3.0:1789760762.687119:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692790574413 time 692804813661 state 0x2 ref 0 00000100:00100000:3.0:1789760762.722520:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760762.741445:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503648640 00000100:00100000:3.0:1789760762.741455:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503648640 00000100:00100000:3.0:1789760762.741521:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692831887493 time 692847491514 state 0x0 ref 1 00000100:00100000:3.0:1789760762.741536:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692847751930 time 692847491514 state 0x1 ref 1 00000100:00100000:3.0:1789760762.741568:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692847751930 time 692866175181 state 0x0 ref 1 00000100:00100000:3.0:1789760762.741579:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503648640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.741625:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72392704, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22868 00010000:00010000:3.0:1789760762.741634:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.741655:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.741659:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.741665:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.741679:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.749598:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72392704 -> 72396800 00002000:00010000:3.0:1789760762.749605:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141416 -> 141424 00010000:00010000:3.0:1789760762.749619:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749632:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749642:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749650:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.749654:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749662:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749688:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749695:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72392704->72396799] (req 72392704->72396799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.749807:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503648640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8224us (8507us total) trans 4294984981 rc 0/0 00000100:00100000:3.0:1789760762.749818:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17685 00000100:00100000:3.0:1789760762.749827:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692847751930 time 692866175181 state 0x2 ref 0 00000001:00080000:3.0:1789760762.762571:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984982 is committed 00000100:00100000:3.0:1789760762.766592:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760762.777293:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984983 is committed 00000100:00100000:3.0:1789760762.784946:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503649024 00000100:00100000:3.0:1789760762.784957:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503649024 00000100:00100000:3.0:1789760762.785010:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692878061537 time 692893171148 state 0x0 ref 1 00000100:00100000:3.0:1789760762.785026:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692893431564 time 692893171148 state 0x1 ref 1 00000100:00100000:3.0:1789760762.785057:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692893431564 time 692909663436 state 0x0 ref 1 00000100:00100000:3.0:1789760762.785341:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503649024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.785411:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72404992, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x22880 00010000:00010000:3.0:1789760762.785421:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.785448:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.785453:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.785459:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.785472:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.791487:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72404992 -> 72409088 00002000:00010000:3.0:1789760762.791608:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760761 -> 1789760762 00002000:00010000:3.0:1789760762.791612:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760761 -> 1789760762 00002000:00010000:3.0:1789760762.791615:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141440 -> 141448 00010000:00010000:3.0:1789760762.791628:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.791641:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.791651:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.791658:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.791663:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.791672:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.791692:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.791701:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eeb64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72404992->72409087] (req 72404992->72409087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.791893:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503649024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6822us (7422us total) trans 4294984984 rc 0/0 00000100:00100000:3.0:1789760762.795286:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17688 00000100:00100000:3.0:1789760762.795298:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692893431564 time 692909663436 state 0x2 ref 0 00000100:00100000:3.0:1789760762.804030:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760762.811960:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984985 is committed 00000100:00100000:3.0:1789760762.826425:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503649408 00000100:00100000:3.0:1789760762.826435:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503649408 00000100:00100000:3.0:1789760762.826486:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692929846184 time 692941399238 state 0x0 ref 1 00000100:00100000:3.0:1789760762.826567:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692941659654 time 692941399238 state 0x1 ref 1 00000100:00100000:3.0:1789760762.826599:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692941659654 time 692951205774 state 0x0 ref 1 00000100:00100000:3.0:1789760762.826612:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503649408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.826668:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72417280, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22898 00010000:00010000:3.0:1789760762.826677:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.826767:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760762.826866:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.826873:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.826887:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.833922:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72417280 -> 72421376 00002000:00010000:3.0:1789760762.833927:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141464 -> 141472 00010000:00010000:3.0:1789760762.833940:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.833953:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.833962:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.833969:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.833974:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.833983:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834210:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834218:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eeb79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72417280->72421375] (req 72417280->72421375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.834322:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503649408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7707us (7995us total) trans 4294984987 rc 0/0 00000100:00100000:3.0:1789760762.834331:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17691 00000100:00100000:3.0:1789760762.834341:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692941659654 time 692951205774 state 0x2 ref 0 00000100:00100000:3.0:1789760762.840233:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760762.865831:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760762.882304:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760762.891011:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72433664 -> 72437760 00002000:00010000:3.0:1789760762.891017:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141496 -> 141504 00010000:00010000:3.0:1789760762.891033:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083eeb95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72433664->72437759] (req 72433664->72437759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891047:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083eeb95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72433664->72437759] (req 72433664->72437759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891057:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083eeb95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72433664->72437759] (req 72433664->72437759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891065:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760762.891071:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083eeb95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72433664->72437759] (req 72433664->72437759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891080:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083eeb95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72433664->72437759] (req 72433664->72437759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891101:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083eeb95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72433664->72437759] (req 72433664->72437759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891108:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083eeb95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72433664->72437759] (req 72433664->72437759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.891303:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503649920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8740us (9004us total) trans 4294984991 rc 0/0 00000100:00100000:3.0:1789760762.891315:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17695 00000100:00100000:3.0:1789760762.891325:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 692990883814 time 693007153893 state 0x2 ref 0 00000100:00100000:3.0:1789760762.953678:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503650432 00000100:00100000:3.0:1789760762.953688:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503650432 00000100:00100000:3.0:1789760762.953748:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693049110633 time 693063460849 state 0x0 ref 1 00000100:00100000:3.0:1789760762.953828:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693063721265 time 693063460849 state 0x1 ref 1 00000100:00100000:3.0:1789760762.953898:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693063721265 time 693078503255 state 0x0 ref 1 00000100:00100000:3.0:1789760762.953911:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503650432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760762.953954:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72450048, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x228d8 00010000:00010000:3.0:1789760762.953964:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eebb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.953980:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760762.953984:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.953991:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eebb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72450048->72454143] (req 72450048->72454143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.954004:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eebb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72450048->72454143] (req 72450048->72454143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.972749:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984996 is committed 00000100:00100000:3.0:1789760762.984656:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760762.989921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984997 is committed 00000001:00080000:3.0:1789760763.006052:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984998 is committed 00000001:00080000:3.0:1789760763.017123:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984999 is committed 00000100:00100000:3.0:1789760763.026225:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503651072 00000100:00100000:3.0:1789760763.026235:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503651072 00000100:00100000:3.0:1789760763.026294:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693119996402 time 693136451834 state 0x0 ref 1 00000100:00100000:3.0:1789760763.026309:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693136712250 time 693136451834 state 0x1 ref 1 00000100:00100000:3.0:1789760763.026376:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693136712250 time 693150982187 state 0x0 ref 1 00000100:00100000:3.0:1789760763.026415:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503651072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.026657:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72470528, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22900 00010000:00010000:3.0:1789760763.026667:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eebd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.026684:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.026688:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.026694:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eebd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72470528->72474623] (req 72470528->72474623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.026718:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eebd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72470528->72474623] (req 72470528->72474623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.031639:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985000 is committed 00000100:00100000:3.0:1789760763.035221:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503651200 00000100:00100000:3.0:1789760763.035231:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503651200 00000100:00100000:3.0:1789760763.035282:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693136712250 time 693150982187 state 0x0 ref 1 00000100:00100000:3.0:1789760763.035297:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693151242603 time 693150982187 state 0x1 ref 1 00000100:00100000:3.0:1789760763.035358:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693151242603 time 693159963993 state 0x0 ref 1 00000100:00100000:3.0:1789760763.035370:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503651200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.035434:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72474624, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22908 00010000:00010000:3.0:1789760763.035454:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.035474:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.035478:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.035484:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.035498:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.040107:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72474624 -> 72478720 00002000:00010000:3.0:1789760763.040113:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141576 -> 141584 00010000:00010000:3.0:1789760763.040126:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.040139:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.040148:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.040155:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.040160:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.040168:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.040186:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.040193:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebdb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72474624->72478719] (req 72474624->72478719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.040327:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503651200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4954us (5195us total) trans 4294985001 rc 0/0 00000100:00100000:3.0:1789760763.040338:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17705 00000100:00100000:3.0:1789760763.040348:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693151242603 time 693159963993 state 0x2 ref 0 00000100:00100000:3.0:1789760763.067338:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503651584 00000100:00100000:3.0:1789760763.067348:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503651584 00000100:00100000:3.0:1789760763.067443:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693173551973 time 693181757030 state 0x0 ref 1 00000100:00100000:3.0:1789760763.067458:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693182017446 time 693181757030 state 0x1 ref 1 00000100:00100000:3.0:1789760763.067485:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693182017446 time 693192092406 state 0x0 ref 1 00000100:00100000:3.0:1789760763.067497:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503651584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.067542:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72486912, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22920 00010000:00010000:3.0:1789760763.067551:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.067566:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.067571:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.067577:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.067590:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.076964:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72486912 -> 72491008 00002000:00010000:3.0:1789760763.076970:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141600 -> 141608 00010000:00010000:3.0:1789760763.076983:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.076996:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.077006:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.077014:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.077018:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.077027:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.077048:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.077055:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72486912->72491007] (req 72486912->72491007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.077174:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503651584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9673us (10016us total) trans 4294985004 rc 0/0 00000100:00100000:3.0:1789760763.077185:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17708 00000100:00100000:3.0:1789760763.077194:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693182017446 time 693192092406 state 0x2 ref 0 00000100:00100000:3.0:1789760763.082464:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503651712 00000100:00100000:3.0:1789760763.082474:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503651712 00000100:00100000:3.0:1789760763.082527:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693182017446 time 693192092406 state 0x0 ref 1 00000100:00100000:3.0:1789760763.082541:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693192352822 time 693192092406 state 0x1 ref 1 00000100:00100000:3.0:1789760763.082620:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693192352822 time 693207225781 state 0x0 ref 1 00000100:00100000:3.0:1789760763.082633:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503651712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.082678:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72491008, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22928 00010000:00010000:3.0:1789760763.082688:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.082703:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.082717:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.082723:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.082736:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.088314:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72491008 -> 72495104 00002000:00010000:3.0:1789760763.088320:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141608 -> 141616 00010000:00010000:3.0:1789760763.088334:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.088346:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.088356:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.088364:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.088368:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.088378:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.088417:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.088426:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eebf7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72491008->72495103] (req 72491008->72495103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.088580:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503651712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5944us (6351us total) trans 4294985005 rc 0/0 00000100:00100000:3.0:1789760763.088591:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17709 00000100:00100000:3.0:1789760763.088600:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693192352822 time 693207225781 state 0x2 ref 0 00000001:00080000:3.0:1789760763.098606:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985006 is committed 00000100:00100000:3.0:1789760763.120781:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503652096 00000100:00100000:3.0:1789760763.120790:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503652096 00000100:00100000:3.0:1789760763.120852:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693218552869 time 693231701809 state 0x0 ref 1 00000100:00100000:3.0:1789760763.120867:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693231962225 time 693231701809 state 0x1 ref 1 00000100:00100000:3.0:1789760763.120887:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693231962225 time 693245493964 state 0x0 ref 1 00000100:00100000:3.0:1789760763.120899:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503652096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.120954:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72503296, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22940 00010000:00010000:3.0:1789760763.120964:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.120979:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.120984:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.121007:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.121023:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.125146:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72503296 -> 72507392 00002000:00010000:3.0:1789760763.125152:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141632 -> 141640 00010000:00010000:3.0:1789760763.125226:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.125246:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.125257:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.125264:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.125269:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.125279:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.125298:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.125305:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72503296->72507391] (req 72503296->72507391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.125510:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503652096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4609us (5056us total) trans 4294985008 rc 0/0 00000100:00100000:3.0:1789760763.125521:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17712 00000100:00100000:3.0:1789760763.125531:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693231962225 time 693245493964 state 0x2 ref 0 00000100:00100000:3.0:1789760763.133416:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503652224 00000100:00100000:3.0:1789760763.133426:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503652224 00000100:00100000:3.0:1789760763.133492:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693231962225 time 693245493964 state 0x0 ref 1 00000100:00100000:3.0:1789760763.133508:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693245754380 time 693245493964 state 0x1 ref 1 00000100:00100000:3.0:1789760763.133540:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693245754380 time 693258146161 state 0x0 ref 1 00000100:00100000:3.0:1789760763.133552:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503652224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.133599:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72507392, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22948 00010000:00010000:3.0:1789760763.133607:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.133623:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.133628:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.133635:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.133650:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.139132:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72507392 -> 72511488 00002000:00010000:3.0:1789760763.139138:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141640 -> 141648 00010000:00010000:3.0:1789760763.139152:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.139164:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.139179:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.139185:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.139190:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.139198:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.139217:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.139225:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72507392->72511487] (req 72507392->72511487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.139327:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503652224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5772us (6063us total) trans 4294985009 rc 0/0 00000100:00100000:3.0:1789760763.139336:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17713 00000100:00100000:3.0:1789760763.139346:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693245754380 time 693258146161 state 0x2 ref 0 00000100:00100000:3.0:1789760763.159351:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503652480 00000100:00100000:3.0:1789760763.159361:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503652480 00000100:00100000:3.0:1789760763.159434:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693258406577 time 693270707327 state 0x0 ref 1 00000100:00100000:3.0:1789760763.159460:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693270967743 time 693270707327 state 0x1 ref 1 00000100:00100000:3.0:1789760763.159489:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693270967743 time 693284095312 state 0x0 ref 1 00000100:00100000:3.0:1789760763.159502:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503652480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.159548:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72515584, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22958 00010000:00010000:3.0:1789760763.159557:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159573:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.159577:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.159583:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159596:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.165701:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72515584 -> 72519680 00002000:00010000:3.0:1789760763.166067:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141656 -> 141664 00010000:00010000:3.0:1789760763.166137:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.166151:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.166160:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.166173:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.166178:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.166186:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.166205:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.166214:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72515584->72519679] (req 72515584->72519679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.174742:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503652608 00000100:00100000:3.0:1789760763.174752:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503652608 00000100:00100000:3.0:1789760763.174876:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693270967743 time 693284095312 state 0x0 ref 1 00000100:00100000:3.0:1789760763.174892:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693284355728 time 693284095312 state 0x1 ref 1 00000100:00100000:3.0:1789760763.175012:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693284355728 time 693299617631 state 0x0 ref 1 00000100:00100000:3.0:1789760763.175026:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503652608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.175082:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72519680, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22960 00010000:00010000:3.0:1789760763.175090:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.175118:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.175123:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.175129:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.175143:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.182188:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72519680 -> 72523776 00002000:00010000:3.0:1789760763.182198:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141664 -> 141672 00010000:00010000:3.0:1789760763.182212:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.182225:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.182234:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.182241:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.182246:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.182259:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.182290:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.182298:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72519680->72523775] (req 72519680->72523775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.182439:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503652608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7410us (7773us total) trans 4294985012 rc 0/0 00000100:00100000:3.0:1789760763.182449:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17716 00000100:00100000:3.0:1789760763.182458:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693284355728 time 693299617631 state 0x2 ref 0 00000100:00100000:3.0:1789760763.189198:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503652736 00000100:00100000:3.0:1789760763.189208:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503652736 00000100:00100000:3.0:1789760763.189260:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693284355728 time 693299617631 state 0x0 ref 1 00000100:00100000:3.0:1789760763.189275:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693299878047 time 693299617631 state 0x1 ref 1 00000100:00100000:3.0:1789760763.189306:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693299878047 time 693313912753 state 0x0 ref 1 00000100:00100000:3.0:1789760763.189319:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503652736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.189367:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72523776, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22968 00010000:00010000:3.0:1789760763.189377:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.189446:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.189452:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.189458:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.189472:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.194103:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72523776 -> 72527872 00002000:00010000:3.0:1789760763.194109:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141672 -> 141680 00010000:00010000:3.0:1789760763.194122:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.194135:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.194144:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.194151:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.194155:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.194174:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.194194:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.194206:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72523776->72527871] (req 72523776->72527871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.194322:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503652736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5000us (5256us total) trans 4294985013 rc 0/0 00000100:00100000:3.0:1789760763.194332:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17717 00000100:00100000:3.0:1789760763.194343:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693299878047 time 693313912753 state 0x2 ref 0 00000100:00100000:3.0:1789760763.229914:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503653120 00000100:00100000:3.0:1789760763.229924:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503653120 00000100:00100000:3.0:1789760763.229978:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693323187004 time 693339382355 state 0x0 ref 1 00000100:00100000:3.0:1789760763.229994:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693339642771 time 693339382355 state 0x1 ref 1 00000100:00100000:3.0:1789760763.230026:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693339642771 time 693354632725 state 0x0 ref 1 00000100:00100000:3.0:1789760763.230038:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503653120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.230086:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72536064, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22980 00010000:00010000:3.0:1789760763.230100:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.230116:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.230121:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.230127:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72536064->72540159] (req 72536064->72540159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.230147:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eec44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72536064->72540159] (req 72536064->72540159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.244375:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503653248 00000100:00100000:3.0:1789760763.244385:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503653248 00000100:00100000:3.0:1789760763.244551:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693339642771 time 693354632725 state 0x0 ref 1 00000100:00100000:3.0:1789760763.244565:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693354893141 time 693354632725 state 0x1 ref 1 00000100:00100000:3.0:1789760763.244586:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693354893141 time 693369193301 state 0x0 ref 1 00000100:00100000:3.0:1789760763.244598:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503653248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.244643:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72540160, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22988 00010000:00010000:3.0:1789760763.244652:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.244668:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.244672:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.244679:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72540160->72544255] (req 72540160->72544255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.244694:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eec4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72540160->72544255] (req 72540160->72544255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.303138:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985021 is committed 00000001:00080000:3.0:1789760763.335899:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985023 is committed 00000100:00100000:3.0:1789760763.343356:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760763.366292:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760763.380248:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760763.388963:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760763.393872:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72581120 -> 72585216 00002000:00010000:3.0:1789760763.393878:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141784 -> 141792 00010000:00010000:3.0:1789760763.393891:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083eec91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72581120->72585215] (req 72581120->72585215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.393903:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083eec91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72581120->72585215] (req 72581120->72585215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.393910:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083eec91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72581120->72585215] (req 72581120->72585215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.393916:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.393921:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083eec91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72581120->72585215] (req 72581120->72585215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.393930:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083eec91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72581120->72585215] (req 72581120->72585215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.393948:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083eec91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72581120->72585215] (req 72581120->72585215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.393955:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083eec91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72581120->72585215] (req 72581120->72585215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.394083:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503654528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4913us (5128us total) trans 4294985027 rc 0/0 00000100:00100000:3.0:1789760763.394094:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17731 00000100:00100000:3.0:1789760763.394103:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693505468581 time 693513760928 state 0x2 ref 0 00000100:00100000:3.0:1789760763.397360:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503654656 00000100:00100000:3.0:1789760763.397369:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503654656 00000100:00100000:3.0:1789760763.397450:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693505468581 time 693513760928 state 0x0 ref 1 00000100:00100000:3.0:1789760763.397461:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693514021344 time 693513760928 state 0x1 ref 1 00000100:00100000:3.0:1789760763.397478:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693514021344 time 693522085679 state 0x0 ref 1 00000100:00100000:3.0:1789760763.397487:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503654656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.397523:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72585216, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x229e0 00010000:00010000:3.0:1789760763.397530:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.397543:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.397546:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.397551:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.397560:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.401817:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72585216 -> 72589312 00002000:00010000:3.0:1789760763.401823:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141792 -> 141800 00010000:00010000:3.0:1789760763.401835:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401848:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401857:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401865:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.401870:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401880:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401901:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401909:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eec98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72585216->72589311] (req 72585216->72589311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.402043:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503654656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4551us (4763us total) trans 4294985028 rc 0/0 00000100:00100000:3.0:1789760763.402054:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17732 00000100:00100000:3.0:1789760763.402064:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693514021344 time 693522085679 state 0x2 ref 0 00010000:00010000:3.0:1789760763.410537:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760763.447372:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503655040 00000100:00100000:3.0:1789760763.447382:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503655040 00000100:00100000:3.0:1789760763.447478:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693530610788 time 693547604299 state 0x0 ref 1 00000100:00100000:3.0:1789760763.447493:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693547864715 time 693547604299 state 0x1 ref 1 00000100:00100000:3.0:1789760763.447514:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693547864715 time 693572120952 state 0x0 ref 1 00000100:00100000:3.0:1789760763.447526:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503655040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.447571:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72597504, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x229f8 00010000:00010000:3.0:1789760763.447580:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.447596:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.447600:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.447606:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.447619:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.452920:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72597504 -> 72601600 00002000:00010000:3.0:1789760763.452928:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141816 -> 141824 00010000:00010000:3.0:1789760763.452942:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.452955:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.452963:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.452971:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.452975:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.452985:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.453005:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.453013:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72597504->72601599] (req 72597504->72601599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.453298:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503655040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5767us (6010us total) trans 4294985031 rc 0/0 00000100:00100000:3.0:1789760763.453309:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17735 00000100:00100000:3.0:1789760763.453319:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693547864715 time 693572120952 state 0x2 ref 0 00000100:00100000:3.0:1789760763.461862:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760763.503648:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985034 is committed 00000100:00100000:3.0:1789760763.508945:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503655552 00000100:00100000:3.0:1789760763.508956:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503655552 00000100:00100000:3.0:1789760763.509010:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693597651642 time 693614957873 state 0x0 ref 1 00000100:00100000:3.0:1789760763.509024:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693615218289 time 693614957873 state 0x1 ref 1 00000100:00100000:3.0:1789760763.509052:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693615218289 time 693633658681 state 0x0 ref 1 00000100:00100000:3.0:1789760763.509065:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503655552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.509112:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72613888, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22a18 00010000:00010000:3.0:1789760763.509122:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.509138:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.509143:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.509149:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.509163:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.517647:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72613888 -> 72617984 00002000:00010000:3.0:1789760763.517653:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141848 -> 141856 00010000:00010000:3.0:1789760763.517667:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.517682:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.517692:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.517700:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.517714:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.517725:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.517851:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.517861:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eecc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72613888->72617983] (req 72613888->72617983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.518235:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503655552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9167us (9429us total) trans 4294985035 rc 0/0 00000100:00100000:3.0:1789760763.518245:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17739 00000100:00100000:3.0:1789760763.518255:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693615218289 time 693633658681 state 0x2 ref 0 00002000:00010000:3.0:1789760763.533699:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72617984 -> 72622080 00002000:00010000:3.0:1789760763.533720:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141856 -> 141864 00010000:00010000:3.0:1789760763.533733:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eecd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72617984->72622079] (req 72617984->72622079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.533809:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eecd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72617984->72622079] (req 72617984->72622079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.533820:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eecd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72617984->72622079] (req 72617984->72622079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.533827:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.533832:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eecd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72617984->72622079] (req 72617984->72622079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.533842:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eecd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72617984->72622079] (req 72617984->72622079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.533862:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eecd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72617984->72622079] (req 72617984->72622079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.533870:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083eecd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72617984->72622079] (req 72617984->72622079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.533962:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503655680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8152us (8948us total) trans 4294985036 rc 0/0 00000100:00100000:3.0:1789760763.533973:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17740 00000100:00100000:3.0:1789760763.533982:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693633919097 time 693650342171 state 0x2 ref 0 00000100:00100000:3.0:1789760763.570006:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503656064 00000100:00100000:3.0:1789760763.570018:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503656064 00000100:00100000:3.0:1789760763.570075:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693663836009 time 693677883406 state 0x0 ref 1 00000100:00100000:3.0:1789760763.570090:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693678143822 time 693677883406 state 0x1 ref 1 00000100:00100000:3.0:1789760763.570123:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693678143822 time 693694729616 state 0x0 ref 1 00000100:00100000:3.0:1789760763.570221:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503656064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.570337:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72630272, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22a38 00010000:00010000:3.0:1789760763.570347:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eece5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.570364:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.570369:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.570375:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eece5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72630272->72634367] (req 72630272->72634367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.570438:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eece5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72630272->72634367] (req 72630272->72634367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.588685:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503656192 00000100:00100000:3.0:1789760763.588695:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503656192 00000100:00100000:3.0:1789760763.588777:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693678143822 time 693694729616 state 0x0 ref 1 00000100:00100000:3.0:1789760763.588801:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693694990032 time 693694729616 state 0x1 ref 1 00000100:00100000:3.0:1789760763.588837:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693694990032 time 693713444189 state 0x0 ref 1 00000100:00100000:3.0:1789760763.588851:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503656192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.588899:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72634368, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22a40 00010000:00010000:3.0:1789760763.588908:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.588923:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.588927:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.588933:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.588948:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.594817:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72634368 -> 72638464 00002000:00010000:3.0:1789760763.594822:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141888 -> 141896 00010000:00010000:3.0:1789760763.594835:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.594849:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.594858:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.594870:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.594874:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.594884:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.594970:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.594978:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eecec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72634368->72638463] (req 72634368->72638463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.595128:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503656192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6272us (6821us total) trans 4294985040 rc 0/0 00000100:00100000:3.0:1789760763.595139:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17744 00000100:00100000:3.0:1789760763.595548:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693694990032 time 693713444189 state 0x2 ref 0 00000100:00100000:3.0:1789760763.597930:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760763.604868:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:afc717a1-11ba-4ef0-ab10-103aa692ca98:3638:1876699523601792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.604921:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.605009:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760763.605016:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3638:1876699523601920:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.605044:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.605074:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523601792 00000100:00100000:3.0:1789760763.605081:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523601792 00000100:00100000:3.0:1789760763.605122:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:3.0:1789760763.605130:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3638:x1876699523601792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.605187:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3638:x1876699523601792:12345-0@lo:400:kworker.0 Request processed in 57us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.605194:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:3.0:1789760763.605235:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523601920 00000100:00100000:3.0:1789760763.605247:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523601920 00000100:00100000:3.0:1789760763.605285:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1789760763.605291:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce858380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3638:x1876699523601920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.605322:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce858380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3638:x1876699523601920:12345-0@lo:400:kworker.0 Request processed in 31us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.605328:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1789760763.605364:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3638:1876699523601920:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.605373:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:afc717a1-11ba-4ef0-ab10-103aa692ca98:3638:1876699523601792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.606644:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523602048:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.606693:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.606870:0:3636:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760763.606876:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3636:1876699523602176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.606902:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.606933:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523602048 00000100:00100000:3.0:1789760763.606938:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523602048 00000100:00100000:3.0:1789760763.606974:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:3.0:1789760763.606982:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3636:x1876699523602048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.607028:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3636:x1876699523602048:12345-0@lo:400:kworker.0 Request processed in 47us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.607035:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:3.0:1789760763.607052:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523602176 00000100:00100000:3.0:1789760763.607056:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523602176 00000100:00100000:3.0:1789760763.607081:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1789760763.607088:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3636:x1876699523602176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.607119:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3636:x1876699523602176:12345-0@lo:400:kworker.0 Request processed in 32us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.607125:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1789760763.607170:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3636:1876699523602176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.607180:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523602048:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.607926:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523602304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.607965:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.608038:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760763.608045:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3638:1876699523602432:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.608068:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.608096:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523602304 00000100:00100000:3.0:1789760763.608100:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523602304 00000100:00100000:3.0:1789760763.608128:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:3.0:1789760763.608134:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce859180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523602304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.608535:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce859180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523602304:12345-0@lo:400:kworker.0 Request processed in 400us (569us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.608542:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:3.0:1789760763.608563:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523602432 00000100:00100000:3.0:1789760763.608567:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523602432 00000100:00100000:3.0:1789760763.608601:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 937 00000100:00100000:3.0:1789760763.608608:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce858e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3638:x1876699523602432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.608640:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce858e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3638:x1876699523602432:12345-0@lo:400:kworker.0 Request processed in 33us (571us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.608646:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 937 00000100:00080000:3.0:1789760763.608660:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760763.608664:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760763.608689:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3638:1876699523602432:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.608698:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523602304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.608909:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523601664:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760763.608931:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523602560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.608969:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.609033:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523603072:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.609055:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.609615:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cce858a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523601536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760763.609628:0:3636:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760763.609633:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523602688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.609657:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.609679:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.609902:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760763.609909:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523602816:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.609931:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.609957:0:8429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523602688 00000100:00100000:3.0:1789760763.609961:0:8429:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523602688 00000100:00100000:3.0:1789760763.609985:0:8429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523602816 00000100:00100000:3.0:1789760763.609988:0:8429:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523602816 00000100:00100000:3.0:1789760763.610005:0:8429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:3.0:1789760763.610011:0:8429:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523602688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.610038:0:8429:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523602688:12345-0@lo:400:kworker.0 Request processed in 27us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.610044:0:8429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:3.0:1789760763.610055:0:8429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:3.0:1789760763.610061:0:8429:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523602816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.610086:0:8429:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523602816:12345-0@lo:400:kworker.0 Request processed in 25us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.610090:0:8429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00080000:3.0:1789760763.610118:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760763.610121:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760763.610122:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760763.610124:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760763.612636:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523602816:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.612659:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3639:1876699523602944:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.612717:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523602688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.612731:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523602560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760763.613122:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523603072:0@lo:400:kworker.0 00000001:00080000:3.0:1789760763.635105:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985043 is committed 00000001:00080000:3.0:1789760763.656832:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985044 is committed 00000001:00080000:3.0:1789760763.667379:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985045 is committed 00000100:00100000:3.0:1789760763.693436:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503657088 00000100:00100000:3.0:1789760763.693446:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503657088 00000100:00100000:3.0:1789760763.693500:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693786904753 time 693801141065 state 0x0 ref 1 00000100:00100000:3.0:1789760763.693517:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693801401481 time 693801141065 state 0x1 ref 1 00000100:00100000:3.0:1789760763.693590:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693801401481 time 693818196175 state 0x0 ref 1 00000100:00100000:3.0:1789760763.693604:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503657088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.693660:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72663040, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22a78 00010000:00010000:3.0:1789760763.693670:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.693685:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.693690:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.693697:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.693722:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.704060:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72663040 -> 72667136 00002000:00010000:3.0:1789760763.704066:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141944 -> 141952 00010000:00010000:3.0:1789760763.704079:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.704091:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.704100:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.704107:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.704112:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.704121:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.704139:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.704147:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72663040->72667135] (req 72663040->72667135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.704243:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503657088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10636us (10976us total) trans 4294985047 rc 0/0 00000100:00100000:3.0:1789760763.704253:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17751 00000100:00100000:3.0:1789760763.704264:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693801401481 time 693818196175 state 0x2 ref 0 00000100:00100000:3.0:1789760763.722473:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503657216 00000100:00100000:3.0:1789760763.722485:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503657216 00000100:00100000:3.0:1789760763.722543:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693801401481 time 693818196175 state 0x0 ref 1 00000100:00100000:3.0:1789760763.722559:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693818456591 time 693818196175 state 0x1 ref 1 00000100:00100000:3.0:1789760763.722584:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693818456591 time 693847191458 state 0x0 ref 1 00000100:00100000:3.0:1789760763.722600:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503657216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.722658:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72667136, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22a80 00010000:00010000:3.0:1789760763.722668:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.722683:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.722688:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.722695:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.722719:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.728763:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72667136 -> 72671232 00002000:00010000:3.0:1789760763.728769:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141952 -> 141960 00010000:00010000:3.0:1789760763.728782:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.728796:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.728804:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.728811:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.728816:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.728824:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.728844:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.728853:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72667136->72671231] (req 72667136->72671231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.728954:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503657216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6352us (6830us total) trans 4294985048 rc 0/0 00000100:00100000:3.0:1789760763.728966:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17752 00000100:00100000:3.0:1789760763.728977:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693818456591 time 693847191458 state 0x2 ref 0 00000100:00100000:3.0:1789760763.760530:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503657600 00000100:00100000:3.0:1789760763.760541:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503657600 00000100:00100000:3.0:1789760763.760594:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693857204855 time 693872116250 state 0x0 ref 1 00000100:00100000:3.0:1789760763.760607:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693872376666 time 693872116250 state 0x1 ref 1 00000100:00100000:3.0:1789760763.760636:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693872376666 time 693885243523 state 0x0 ref 1 00000100:00100000:3.0:1789760763.760649:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503657600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.760695:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72679424, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22a98 00010000:00010000:3.0:1789760763.760714:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.760729:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.760733:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.760739:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72679424->72683519] (req 72679424->72683519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.760762:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eed39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72679424->72683519] (req 72679424->72683519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.769316:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985051 is committed 00000100:00100000:3.0:1789760763.773513:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503657728 00000100:00100000:3.0:1789760763.773522:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503657728 00000100:00100000:3.0:1789760763.773573:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693872376666 time 693885243523 state 0x0 ref 1 00000100:00100000:3.0:1789760763.773588:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693885503939 time 693885243523 state 0x1 ref 1 00000100:00100000:3.0:1789760763.773663:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693885503939 time 693898268721 state 0x0 ref 1 00000100:00100000:3.0:1789760763.773676:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503657728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.773731:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72683520, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x22aa0 00010000:00010000:3.0:1789760763.773741:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.773767:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.773771:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.773777:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.773791:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.782119:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72683520 -> 72687616 00002000:00010000:3.0:1789760763.782124:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 141984 -> 141992 00010000:00010000:3.0:1789760763.782137:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.782149:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.782158:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.782166:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.782171:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.782179:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.782199:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.782212:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72683520->72687615] (req 72683520->72687615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.782304:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503657728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8625us (8953us total) trans 4294985052 rc 0/0 00000100:00100000:3.0:1789760763.782315:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17756 00000100:00100000:3.0:1789760763.782324:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693885503939 time 693898268721 state 0x2 ref 0 00000001:00080000:3.0:1789760763.804998:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985054 is committed 00000100:00100000:3.0:1789760763.818574:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503658112 00000100:00100000:3.0:1789760763.818585:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503658112 00000100:00100000:3.0:1789760763.818640:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693910987379 time 693919905247 state 0x0 ref 1 00000100:00100000:3.0:1789760763.818655:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693920165663 time 693919905247 state 0x1 ref 1 00000100:00100000:3.0:1789760763.818680:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693920165663 time 693943287283 state 0x0 ref 1 00000100:00100000:3.0:1789760763.818693:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503658112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.818756:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72695808, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22ab8 00010000:00010000:3.0:1789760763.818765:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.818779:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.818783:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.818789:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.818804:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.824074:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72695808 -> 72699904 00002000:00010000:3.0:1789760763.824079:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142008 -> 142016 00010000:00010000:3.0:1789760763.824092:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824103:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824112:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824118:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.824123:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824131:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824150:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.824157:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eed55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72695808->72699903] (req 72695808->72699903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.824249:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503658112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5553us (6342us total) trans 4294985055 rc 0/0 00000100:00100000:3.0:1789760763.824260:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17759 00000100:00100000:3.0:1789760763.824270:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693920165663 time 693943287283 state 0x2 ref 0 00000100:00100000:3.0:1789760763.833560:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503658240 00000100:00100000:3.0:1789760763.833569:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503658240 00000100:00100000:3.0:1789760763.833625:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693920165663 time 693943287283 state 0x0 ref 1 00000100:00100000:3.0:1789760763.833639:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693943547699 time 693943287283 state 0x1 ref 1 00000100:00100000:3.0:1789760763.833715:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693943547699 time 693958320851 state 0x0 ref 1 00000100:00100000:3.0:1789760763.833728:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503658240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.833841:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72699904, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22ac0 00010000:00010000:3.0:1789760763.833850:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833866:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760763.833870:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.833876:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833890:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.840614:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72699904 -> 72704000 00002000:00010000:3.0:1789760763.840620:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142016 -> 142024 00010000:00010000:3.0:1789760763.840635:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.840647:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.840655:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.840662:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760763.840667:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.840676:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.840696:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.840713:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72699904->72703999] (req 72699904->72703999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.840879:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503658240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7148us (7536us total) trans 4294985056 rc 0/0 00000100:00100000:3.0:1789760763.840892:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17760 00000100:00100000:3.0:1789760763.840902:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693943547699 time 693958320851 state 0x2 ref 0 00000001:00080000:3.0:1789760763.852187:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985057 is committed 00000100:00100000:3.0:1789760763.879194:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503658624 00000100:00100000:3.0:1789760763.879203:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503658624 00000100:00100000:3.0:1789760763.879252:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693972738656 time 693989258338 state 0x0 ref 1 00000100:00100000:3.0:1789760763.879267:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693989518754 time 693989258338 state 0x1 ref 1 00000100:00100000:3.0:1789760763.879330:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693989518754 time 694003935810 state 0x0 ref 1 00000100:00100000:3.0:1789760763.879342:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503658624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760763.880067:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72712192, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22ad8 00010000:00010000:3.0:1789760763.880079:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.880095:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760763.880100:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.880108:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72712192->72716287] (req 72712192->72716287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.880125:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eed71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72712192->72716287] (req 72712192->72716287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.890308:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503658752 00000100:00100000:3.0:1789760763.890318:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503658752 00000100:00100000:3.0:1789760763.890850:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 693989518754 time 694003935810 state 0x0 ref 1 00000100:00100000:3.0:1789760763.890868:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694004196226 time 694003935810 state 0x1 ref 1 00000001:00080000:3.0:1789760764.004096:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985068 is committed 00000100:00080000:3.0:1789760764.010023:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760764.021970:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985069 is committed 00000100:00100000:3.0:1789760764.032410:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503660032 00000100:00100000:3.0:1789760764.032497:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503660032 00000001:00080000:3.0:1789760764.040621:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985070 is committed 00000001:00080000:3.0:1789760764.054942:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985071 is committed 00000001:00080000:3.0:1789760764.069824:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985072 is committed 00000001:00080000:3.0:1789760764.083689:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985073 is committed 00000001:00080000:3.0:1789760764.102810:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985074 is committed 00000001:00080000:3.0:1789760764.117537:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985075 is committed 00000001:00080000:3.0:1789760764.133674:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985076 is committed 00000001:00080000:3.0:1789760764.144594:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985077 is committed 00000001:00080000:3.0:1789760764.154034:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985078 is committed 00000100:00100000:3.0:1789760764.160461:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503661184 00000100:00100000:3.0:1789760764.160475:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503661184 00000100:00100000:3.0:1789760764.160544:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694263627089 time 694273947482 state 0x0 ref 1 00000100:00100000:3.0:1789760764.160560:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694274207898 time 694273947482 state 0x1 ref 1 00000100:00100000:3.0:1789760764.160601:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694274207898 time 694285207957 state 0x0 ref 1 00000100:00100000:3.0:1789760764.160615:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503661184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.160662:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72794112, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22b78 00010000:00010000:3.0:1789760764.160672:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.160689:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.160694:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.160711:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.160725:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.169628:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72794112 -> 72798208 00002000:00010000:3.0:1789760764.169634:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142200 -> 142208 00010000:00010000:3.0:1789760764.169646:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.169660:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.169670:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.169676:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.169681:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.169690:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.169722:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.169731:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eedfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72794112->72798207] (req 72794112->72798207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.169834:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503661184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9218us (11478us total) trans 4294985079 rc 0/0 00000100:00100000:3.0:1789760764.169845:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17783 00000100:00100000:3.0:1789760764.169855:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694274207898 time 694285207957 state 0x2 ref 0 00000001:00080000:3.0:1789760764.208683:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985082 is committed 00000100:00100000:3.0:1789760764.215964:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503661696 00000100:00100000:3.0:1789760764.215974:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503661696 00000100:00100000:3.0:1789760764.216026:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694317377882 time 694328212817 state 0x0 ref 1 00000100:00100000:3.0:1789760764.216040:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694328473233 time 694328212817 state 0x1 ref 1 00000100:00100000:3.0:1789760764.216115:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694328473233 time 694340720986 state 0x0 ref 1 00000100:00100000:3.0:1789760764.216128:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503661696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.216170:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72810496, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22b98 00010000:00010000:3.0:1789760764.216180:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.216200:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.216204:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.216210:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.216223:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.220433:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72810496 -> 72814592 00002000:00010000:3.0:1789760764.220439:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142232 -> 142240 00010000:00010000:3.0:1789760764.220450:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220462:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220471:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220477:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.220482:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220489:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220515:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.220524:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72810496->72814591] (req 72810496->72814591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.220613:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503661696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4483us (4752us total) trans 4294985083 rc 0/0 00000100:00100000:3.0:1789760764.220623:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17787 00000100:00100000:3.0:1789760764.220632:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694328473233 time 694340720986 state 0x2 ref 0 00000100:00100000:3.0:1789760764.263986:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503662208 00000100:00100000:3.0:1789760764.263996:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503662208 00000100:00100000:3.0:1789760764.264054:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694365637419 time 694376294985 state 0x0 ref 1 00000100:00100000:3.0:1789760764.264069:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694376555401 time 694376294985 state 0x1 ref 1 00000100:00100000:3.0:1789760764.264151:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694376555401 time 694388757138 state 0x0 ref 1 00000100:00100000:3.0:1789760764.264163:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503662208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.264206:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72826880, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22bb8 00010000:00010000:3.0:1789760764.264216:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264232:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.264236:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.264243:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264256:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.269236:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72826880 -> 72830976 00002000:00010000:3.0:1789760764.269241:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142264 -> 142272 00010000:00010000:3.0:1789760764.269255:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.269266:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.269273:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.269280:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.269284:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.269293:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.269312:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.269324:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72826880->72830975] (req 72826880->72830975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.269579:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503662208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5411us (5698us total) trans 4294985087 rc 0/0 00000100:00100000:3.0:1789760764.269590:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17791 00000100:00100000:3.0:1789760764.269600:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694376555401 time 694388757138 state 0x2 ref 0 00000100:00100000:3.0:1789760764.316875:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503662720 00000100:00100000:3.0:1789760764.316885:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503662720 00000100:00100000:3.0:1789760764.316945:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694414368945 time 694432246880 state 0x0 ref 1 00000100:00100000:3.0:1789760764.316961:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694432507296 time 694432246880 state 0x1 ref 1 00000100:00100000:3.0:1789760764.316993:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694432507296 time 694441599065 state 0x0 ref 1 00000100:00100000:3.0:1789760764.317006:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503662720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.317055:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72843264, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22bd8 00010000:00010000:3.0:1789760764.317065:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.317082:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.317087:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.317093:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.317112:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.322895:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72843264 -> 72847360 00002000:00010000:3.0:1789760764.322901:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142296 -> 142304 00010000:00010000:3.0:1789760764.322914:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.322927:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.322936:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.322942:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.322947:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.322956:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.322976:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.322984:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72843264->72847359] (req 72843264->72847359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.323081:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503662720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6071us (6297us total) trans 4294985091 rc 0/0 00000100:00100000:3.0:1789760764.323093:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17795 00000100:00100000:3.0:1789760764.323106:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694432507296 time 694441599065 state 0x2 ref 0 00000100:00100000:3.0:1789760764.347918:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503663104 00000100:00100000:3.0:1789760764.347928:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503663104 00000100:00100000:3.0:1789760764.347982:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694452841693 time 694462297616 state 0x0 ref 1 00000100:00100000:3.0:1789760764.347997:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694462558032 time 694462297616 state 0x1 ref 1 00000100:00100000:3.0:1789760764.348030:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694462558032 time 694472636820 state 0x0 ref 1 00000100:00100000:3.0:1789760764.348042:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503663104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.348087:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72855552, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22bf0 00010000:00010000:3.0:1789760764.348095:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.348111:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.348116:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.348122:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.348135:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.353864:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72855552 -> 72859648 00002000:00010000:3.0:1789760764.353870:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142320 -> 142328 00010000:00010000:3.0:1789760764.353884:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.353896:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.353905:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.353912:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.353917:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.353927:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.353947:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.353955:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eee66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72855552->72859647] (req 72855552->72859647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.354088:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503663104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6044us (6796us total) trans 4294985094 rc 0/0 00000100:00100000:3.0:1789760764.354098:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17798 00000100:00100000:3.0:1789760764.354108:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694462558032 time 694472636820 state 0x2 ref 0 00000100:00100000:3.0:1789760764.357179:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503663232 00000100:00100000:3.0:1789760764.357188:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503663232 00000100:00100000:3.0:1789760764.357240:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694462558032 time 694472636820 state 0x0 ref 1 00000100:00100000:3.0:1789760764.357256:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694472897236 time 694472636820 state 0x1 ref 1 00000100:00100000:3.0:1789760764.357289:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694472897236 time 694481895264 state 0x0 ref 1 00000100:00100000:3.0:1789760764.357303:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503663232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.357351:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72859648, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22bf8 00010000:00010000:3.0:1789760764.357360:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.357375:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.357379:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.357386:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.357587:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.362998:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72859648 -> 72863744 00002000:00010000:3.0:1789760764.363004:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142328 -> 142336 00010000:00010000:3.0:1789760764.363018:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.363030:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.363039:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.363047:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.363052:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.363061:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.363078:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.363086:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72859648->72863743] (req 72859648->72863743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.363180:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503663232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5874us (6087us total) trans 4294985095 rc 0/0 00000100:00100000:3.0:1789760764.363190:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17799 00000100:00100000:3.0:1789760764.363199:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694472897236 time 694481895264 state 0x2 ref 0 00000100:00100000:3.0:1789760764.388905:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503663616 00000100:00100000:3.0:1789760764.388919:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503663616 00000100:00100000:3.0:1789760764.388976:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694492969704 time 694503951880 state 0x0 ref 1 00000100:00100000:3.0:1789760764.388992:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694504212296 time 694503951880 state 0x1 ref 1 00000100:00100000:3.0:1789760764.389026:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694504212296 time 694513632173 state 0x0 ref 1 00000100:00100000:3.0:1789760764.389038:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503663616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.389087:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72871936, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x22c10 00010000:00010000:3.0:1789760764.389096:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.389112:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.389116:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.389123:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72871936->72876031] (req 72871936->72876031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.389137:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083eee82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72871936->72876031] (req 72871936->72876031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.402404:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503663744 00000100:00100000:3.0:1789760764.402419:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503663744 00010000:00010000:3.0:1789760764.435729:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760764.464328:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985104 is committed 00000001:00080000:3.0:1789760764.481368:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985105 is committed 00000001:00080000:3.0:1789760764.498989:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985106 is committed 00000001:00080000:3.0:1789760764.514010:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985107 is committed 00000100:00100000:3.0:1789760764.525118:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760764.603366:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760764.662208:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985113 is committed 00000100:00100000:3.0:1789760764.678508:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760764.686686:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985114 is committed 00000100:00100000:3.0:1789760764.693636:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760764.701087:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985115 is committed 00000100:00100000:3.0:1789760764.729434:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760764.814135:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503666688 00000100:00100000:3.0:1789760764.814152:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503666688 00000100:00100000:3.0:1789760764.814235:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694911349713 time 694922946508 state 0x0 ref 1 00000100:00100000:3.0:1789760764.814254:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694923206924 time 694922946508 state 0x1 ref 1 00000100:00100000:3.0:1789760764.815926:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694923206924 time 694940515265 state 0x0 ref 1 00000100:00100000:3.0:1789760764.815967:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503666688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.816044:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72970240, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22cd0 00010000:00010000:3.0:1789760764.816055:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.816075:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.816081:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.816090:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.816106:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.827899:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72970240 -> 72974336 00002000:00010000:3.0:1789760764.827906:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142544 -> 142552 00010000:00010000:3.0:1789760764.827923:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.827936:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.827945:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.827953:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.827958:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.827968:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.827993:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.828001:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72970240->72974335] (req 72970240->72974335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.828140:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503666688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12185us (15004us total) trans 4294985122 rc 0/0 00000100:00100000:3.0:1789760764.828152:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17826 00000100:00100000:3.0:1789760764.828165:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694923206924 time 694940515265 state 0x2 ref 0 00000100:00100000:3.0:1789760764.860194:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503666944 00000100:00100000:3.0:1789760764.860215:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503666944 00000100:00100000:3.0:1789760764.860312:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694940775681 time 694956059037 state 0x0 ref 1 00000100:00100000:3.0:1789760764.860332:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694956319453 time 694956059037 state 0x1 ref 1 00000100:00100000:3.0:1789760764.860490:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694956319453 time 694985094813 state 0x0 ref 1 00000100:00100000:3.0:1789760764.860507:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503666944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.860572:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72978432, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22ce0 00010000:00010000:3.0:1789760764.860583:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.860602:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760764.860607:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.860615:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.860632:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.870207:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72978432 -> 72982528 00002000:00010000:3.0:1789760764.870213:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142560 -> 142568 00010000:00010000:3.0:1789760764.870232:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.870247:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.870256:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.870266:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.870271:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.870281:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.870306:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.870315:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72978432->72982527] (req 72978432->72982527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.870511:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503666944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9999us (13889us total) trans 4294985124 rc 0/0 00000100:00100000:3.0:1789760764.870522:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17828 00000100:00100000:3.0:1789760764.870534:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 694956319453 time 694985094813 state 0x2 ref 0 00000100:00100000:3.0:1789760764.914184:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760764.932621:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503667456 00000100:00100000:3.0:1789760764.932641:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503667456 00000100:00100000:3.0:1789760764.932734:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695021409323 time 695039040401 state 0x0 ref 1 00000100:00100000:3.0:1789760764.932755:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695039300817 time 695039040401 state 0x1 ref 1 00000100:00100000:3.0:1789760764.932791:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695039300817 time 695057397289 state 0x0 ref 1 00000100:00100000:3.0:1789760764.932804:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503667456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760764.932920:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 72994816, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22d00 00010000:00010000:3.0:1789760764.932932:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.932951:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760764.932957:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.932965:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.932981:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.940876:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 72994816 -> 72998912 00002000:00010000:3.0:1789760764.940888:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142592 -> 142600 00010000:00010000:3.0:1789760764.940907:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.940927:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.940938:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.940947:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.940953:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.940964:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.940990:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.940999:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [72994816->72998911] (req 72994816->72998911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.941322:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503667456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8510us (8805us total) trans 4294985128 rc 0/0 00000100:00100000:3.0:1789760764.941336:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17832 00000100:00100000:3.0:1789760764.941349:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695039300817 time 695057397289 state 0x2 ref 0 00000100:00100000:3.0:1789760764.965236:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760764.977630:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73003008 -> 73007104 00002000:00010000:3.0:1789760764.977638:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142608 -> 142616 00010000:00010000:3.0:1789760764.977655:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083eef62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73003008->73007103] (req 73003008->73007103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977672:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083eef62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73003008->73007103] (req 73003008->73007103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977682:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083eef62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73003008->73007103] (req 73003008->73007103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977702:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760764.977708:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083eef62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73003008->73007103] (req 73003008->73007103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977719:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083eef62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73003008->73007103] (req 73003008->73007103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977744:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083eef62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73003008->73007103] (req 73003008->73007103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977764:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083eef62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73003008->73007103] (req 73003008->73007103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.977894:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503667712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7550us (12663us total) trans 4294985130 rc 0/0 00000100:00100000:3.0:1789760764.977907:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17834 00000100:00100000:3.0:1789760764.977923:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695072856549 time 695094927779 state 0x2 ref 0 00000100:00100000:3.0:1789760765.007325:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503667968 00000100:00100000:3.0:1789760765.007340:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503667968 00000100:00100000:3.0:1789760765.007508:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695095188195 time 695115153052 state 0x0 ref 1 00000100:00100000:3.0:1789760765.007528:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695115413468 time 695115153052 state 0x1 ref 1 00000100:00100000:3.0:1789760765.007621:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695115413468 time 695132178010 state 0x0 ref 1 00000100:00100000:3.0:1789760765.007637:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503667968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.007713:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73011200, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22d20 00010000:00010000:3.0:1789760765.007725:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.007747:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.007753:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.007761:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.007779:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.017559:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73011200 -> 73015296 00002000:00010000:3.0:1789760765.017564:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142624 -> 142632 00010000:00010000:3.0:1789760765.017579:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.017597:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.017606:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.017613:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.017618:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.017629:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.017653:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.017661:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73011200->73015295] (req 73011200->73015295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.017778:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503667968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10140us (10647us total) trans 4294985132 rc 0/0 00000100:00100000:3.0:1789760765.017789:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17836 00000100:00100000:3.0:1789760765.017799:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695115413468 time 695132178010 state 0x2 ref 0 00000100:00100000:3.0:1789760765.027176:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760765.084517:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503668480 00000100:00100000:3.0:1789760765.084527:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503668480 00000100:00100000:3.0:1789760765.084584:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695171836010 time 695190643456 state 0x0 ref 1 00000100:00100000:3.0:1789760765.084599:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695190903872 time 695190643456 state 0x1 ref 1 00000100:00100000:3.0:1789760765.084671:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695190903872 time 695209277642 state 0x0 ref 1 00000100:00100000:3.0:1789760765.084684:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503668480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.084744:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73027584, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22d40 00010000:00010000:3.0:1789760765.084753:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.084769:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.084774:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.084780:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73027584->73031679] (req 73027584->73031679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.084793:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eef8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73027584->73031679] (req 73027584->73031679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.123890:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73035776 -> 73039872 00002000:00010000:3.0:1789760765.123895:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142672 -> 142680 00010000:00010000:3.0:1789760765.123909:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083eef9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73035776->73039871] (req 73035776->73039871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.123921:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083eef9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73035776->73039871] (req 73035776->73039871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.123930:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083eef9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73035776->73039871] (req 73035776->73039871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.123938:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.123943:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083eef9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73035776->73039871] (req 73035776->73039871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.123952:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083eef9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73035776->73039871] (req 73035776->73039871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.123971:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083eef9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73035776->73039871] (req 73035776->73039871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.123980:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083eef9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73035776->73039871] (req 73035776->73039871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.124073:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503668736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12456us (12717us total) trans 4294985138 rc 0/0 00000100:00100000:3.0:1789760765.124084:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17842 00000100:00100000:3.0:1789760765.124094:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695222431168 time 695236208319 state 0x2 ref 0 00000001:00080000:3.0:1789760765.137921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985139 is committed 00000100:00100000:3.0:1789760765.142315:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760765.142331:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cce85ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523603328:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760765.142386:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760765.142481:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523603328 00000100:00100000:3.0:1789760765.142488:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523603328 00000100:00100000:3.0:1789760765.142534:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1789760765.142543:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523603328:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760765.142565:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3850534912 avail 3618582528 00000100:00100000:3.0:1789760765.142601:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523603328:12345-0@lo:13:osp-pre-0-1.0 Request processed in 60us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760765.142608:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00080000:3.0:1789760765.142640:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760765.142674:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cce85ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523603328:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1789760765.151327:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760765.151382:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8946680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523603200:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760765.185315:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503669120 00000100:00100000:3.0:1789760765.185325:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503669120 00000100:00100000:3.0:1789760765.185380:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695255343855 time 695277930513 state 0x0 ref 1 00000100:00100000:3.0:1789760765.185550:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695278190929 time 695277930513 state 0x1 ref 1 00000100:00100000:3.0:1789760765.185583:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695278190929 time 695310189279 state 0x0 ref 1 00000100:00100000:3.0:1789760765.185595:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503669120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.185642:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73048064, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22d68 00010000:00010000:3.0:1789760765.185651:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eefaf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.185667:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.185671:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.185677:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eefaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73048064->73052159] (req 73048064->73052159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.185700:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083eefaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73048064->73052159] (req 73048064->73052159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.215041:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503669248 00000100:00100000:3.0:1789760765.215055:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503669248 00000100:00100000:3.0:1789760765.215114:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695278190929 time 695310189279 state 0x0 ref 1 00000100:00100000:3.0:1789760765.215129:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695310449695 time 695310189279 state 0x1 ref 1 00000100:00100000:3.0:1789760765.215208:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695310449695 time 695339814192 state 0x0 ref 1 00000100:00100000:3.0:1789760765.215221:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503669248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.215269:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73052160, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22d70 00010000:00010000:3.0:1789760765.215279:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.215295:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.215300:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.215305:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.215317:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.222250:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73052160 -> 73056256 00002000:00010000:3.0:1789760765.222255:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142704 -> 142712 00010000:00010000:3.0:1789760765.222270:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.222283:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.222292:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.222300:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.222305:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.222314:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.222333:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.222340:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73052160->73056255] (req 73052160->73056255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.222707:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503669248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7471us (9254us total) trans 4294985142 rc 0/0 00000100:00100000:3.0:1789760765.222719:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17846 00000100:00100000:3.0:1789760765.222728:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695310449695 time 695339814192 state 0x2 ref 0 00000100:00100000:3.0:1789760765.265797:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503669760 00000100:00100000:3.0:1789760765.265807:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503669760 00000100:00100000:3.0:1789760765.265868:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695366514054 time 695380890101 state 0x0 ref 1 00000100:00100000:3.0:1789760765.265883:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695381150517 time 695380890101 state 0x1 ref 1 00000100:00100000:3.0:1789760765.265913:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695381150517 time 695390519300 state 0x0 ref 1 00000100:00100000:3.0:1789760765.265926:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffb6fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503669760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.265971:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73068544, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22d90 00010000:00010000:3.0:1789760765.265981:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.265997:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.266001:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.266008:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.266022:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.270941:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73068544 -> 73072640 00002000:00010000:3.0:1789760765.270946:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142736 -> 142744 00010000:00010000:3.0:1789760765.270959:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.270973:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.270982:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.270990:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.270995:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.271004:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.271023:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.271031:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083eefd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73068544->73072639] (req 73068544->73072639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.271120:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffb6fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503669760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5192us (5556us total) trans 4294985146 rc 0/0 00000100:00100000:3.0:1789760765.271130:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17850 00000100:00100000:3.0:1789760765.271140:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695381150517 time 695390519300 state 0x2 ref 0 00002000:00010000:3.0:1789760765.318229:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73084928 -> 73089024 00002000:00010000:3.0:1789760765.318234:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142768 -> 142776 00010000:00010000:3.0:1789760765.318247:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083eefee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73084928->73089023] (req 73084928->73089023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.318258:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083eefee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73084928->73089023] (req 73084928->73089023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.318266:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083eefee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73084928->73089023] (req 73084928->73089023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.318274:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.318279:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083eefee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73084928->73089023] (req 73084928->73089023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.318288:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083eefee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73084928->73089023] (req 73084928->73089023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.318307:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083eefee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73084928->73089023] (req 73084928->73089023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.318314:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083eefee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73084928->73089023] (req 73084928->73089023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.318489:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503670272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4870us (5153us total) trans 4294985150 rc 0/0 00000100:00100000:3.0:1789760765.318503:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17854 00000100:00100000:3.0:1789760765.318514:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695428545129 time 695438207528 state 0x2 ref 0 00000100:00100000:3.0:1789760765.352722:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503670784 00000100:00100000:3.0:1789760765.352789:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503670784 00000100:00100000:3.0:1789760765.396110:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760765.413791:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760765.418152:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985160 is committed 00002000:00010000:3.0:1789760765.418617:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73125888 -> 73129984 00002000:00010000:3.0:1789760765.418622:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142848 -> 142856 00010000:00010000:3.0:1789760765.418635:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef034 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73125888->73129983] (req 73125888->73129983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.418649:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef034 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73125888->73129983] (req 73125888->73129983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.418656:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef034 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73125888->73129983] (req 73125888->73129983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.418663:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.418667:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef034 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73125888->73129983] (req 73125888->73129983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.418676:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef034 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73125888->73129983] (req 73125888->73129983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.418706:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef034 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73125888->73129983] (req 73125888->73129983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.418715:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef034 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73125888->73129983] (req 73125888->73129983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.418910:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503671552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4854us (5122us total) trans 4294985160 rc 0/0 00000100:00100000:3.0:1789760765.418921:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17864 00000100:00100000:3.0:1789760765.418931:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695530875301 time 695538646470 state 0x2 ref 0 00000001:00080000:3.0:1789760765.439882:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985162 is committed 00000001:00080000:3.0:1789760765.457159:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985163 is committed 00010000:00010000:3.0:1789760765.458826:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760765.462060:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503672064 00000100:00100000:3.0:1789760765.462070:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503672064 00000100:00100000:3.0:1789760765.462122:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695559961273 time 695572588259 state 0x0 ref 1 00000100:00100000:3.0:1789760765.462137:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695572848675 time 695572588259 state 0x1 ref 1 00000100:00100000:3.0:1789760765.462169:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695572848675 time 695586776081 state 0x0 ref 1 00000100:00100000:3.0:1789760765.462181:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503672064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.462226:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73142272, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22e20 00010000:00010000:3.0:1789760765.462235:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.462250:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.462255:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.462260:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.462273:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.469007:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73142272 -> 73146368 00002000:00010000:3.0:1789760765.469014:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142880 -> 142888 00010000:00010000:3.0:1789760765.469248:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.469261:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.469270:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.469277:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.469282:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.469291:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.469312:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.469326:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef050 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73142272->73146367] (req 73142272->73146367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.469885:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503672064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7699us (7933us total) trans 4294985164 rc 0/0 00000100:00100000:3.0:1789760765.470477:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17868 00000100:00100000:3.0:1789760765.470488:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695572848675 time 695586776081 state 0x2 ref 0 00000001:00080000:3.0:1789760765.512635:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985167 is committed 00000001:00080000:3.0:1789760765.532091:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985168 is committed 00000100:00100000:3.0:1789760765.553912:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760765.581651:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985171 is committed 00000001:00080000:3.0:1789760765.631114:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985174 is committed 00002000:00010000:3.0:1789760765.711917:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73199616 -> 73203712 00002000:00010000:3.0:1789760765.711922:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 142992 -> 143000 00010000:00010000:3.0:1789760765.711934:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef0b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73199616->73203711] (req 73199616->73203711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.711947:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef0b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73199616->73203711] (req 73199616->73203711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.711957:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef0b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73199616->73203711] (req 73199616->73203711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.711965:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.711969:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef0b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73199616->73203711] (req 73199616->73203711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.711979:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef0b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73199616->73203711] (req 73199616->73203711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.711999:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef0b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73199616->73203711] (req 73199616->73203711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.712006:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef0b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73199616->73203711] (req 73199616->73203711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.712134:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503673856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10106us (10382us total) trans 4294985178 rc 0/0 00000100:00100000:3.0:1789760765.712145:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17882 00000100:00100000:3.0:1789760765.712155:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695812067943 time 695826603429 state 0x2 ref 0 00000100:00100000:3.0:1789760765.754241:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503674368 00000100:00100000:3.0:1789760765.754253:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503674368 00000100:00100000:3.0:1789760765.754310:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695856032859 time 695865282215 state 0x0 ref 1 00000100:00100000:3.0:1789760765.754327:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695865542631 time 695865282215 state 0x1 ref 1 00000100:00100000:3.0:1789760765.754354:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695865542631 time 695878960640 state 0x0 ref 1 00000100:00100000:3.0:1789760765.754366:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503674368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.754456:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73216000, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22eb0 00010000:00010000:3.0:1789760765.754465:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.754482:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.754487:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.754494:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.754509:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.760250:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73216000 -> 73220096 00002000:00010000:3.0:1789760765.760255:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143024 -> 143032 00010000:00010000:3.0:1789760765.760267:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.760280:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.760289:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.760297:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.760301:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.760310:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.760334:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.760342:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73216000->73220095] (req 73216000->73220095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.760541:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503674368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6171us (6392us total) trans 4294985182 rc 0/0 00000100:00100000:3.0:1789760765.760551:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17886 00000100:00100000:3.0:1789760765.760560:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695865542631 time 695878960640 state 0x2 ref 0 00000100:00100000:3.0:1789760765.785931:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503674752 00000100:00100000:3.0:1789760765.785941:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503674752 00000100:00100000:3.0:1789760765.786056:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695887901523 time 695897694051 state 0x0 ref 1 00000100:00100000:3.0:1789760765.786072:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695897954467 time 695897694051 state 0x1 ref 1 00000100:00100000:3.0:1789760765.786102:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695897954467 time 695910708970 state 0x0 ref 1 00000100:00100000:3.0:1789760765.786115:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503674752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.786723:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73228288, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x22ec8 00010000:00010000:3.0:1789760765.786793:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.786810:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.786815:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.786822:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.786837:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.795341:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73228288 -> 73232384 00002000:00010000:3.0:1789760765.795345:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760765.795348:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760765.795350:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143048 -> 143056 00010000:00010000:3.0:1789760765.795362:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.795374:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.795383:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.795434:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.795439:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.795449:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.795473:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.795481:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef0e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73228288->73232383] (req 73228288->73232383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.795886:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503674752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9768us (10232us total) trans 4294985185 rc 0/0 00000100:00100000:3.0:1789760765.795896:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17889 00000100:00100000:3.0:1789760765.795905:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695897954467 time 695910708970 state 0x2 ref 0 00000100:00100000:3.0:1789760765.801903:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503674880 00000100:00100000:3.0:1789760765.801912:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503674880 00000100:00100000:3.0:1789760765.801960:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695897954467 time 695910708970 state 0x0 ref 1 00000100:00100000:3.0:1789760765.801973:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695910969386 time 695910708970 state 0x1 ref 1 00000100:00100000:3.0:1789760765.802042:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695910969386 time 695926610911 state 0x0 ref 1 00000100:00100000:3.0:1789760765.802055:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503674880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.802153:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73232384, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x22ed0 00010000:00010000:3.0:1789760765.802162:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.802177:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.802182:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.802188:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.802261:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.811764:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73232384 -> 73236480 00002000:00010000:3.0:1789760765.811769:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143056 -> 143064 00010000:00010000:3.0:1789760765.811781:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.811794:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.811802:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.811808:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.811812:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.811821:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.811841:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.811853:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73232384->73236479] (req 73232384->73236479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.811944:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503674880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9887us (10079us total) trans 4294985186 rc 0/0 00000100:00100000:3.0:1789760765.811954:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17890 00000100:00100000:3.0:1789760765.811963:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695910969386 time 695926610911 state 0x2 ref 0 00000100:00100000:3.0:1789760765.845076:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503675264 00000100:00100000:3.0:1789760765.845085:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503675264 00000100:00100000:3.0:1789760765.845136:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695941374871 time 695956439081 state 0x0 ref 1 00000100:00100000:3.0:1789760765.845151:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695956699497 time 695956439081 state 0x1 ref 1 00000100:00100000:3.0:1789760765.845181:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695956699497 time 695969788363 state 0x0 ref 1 00000100:00100000:3.0:1789760765.845193:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503675264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.845236:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73244672, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x22ee8 00010000:00010000:3.0:1789760765.845246:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.845262:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.845265:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.845272:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73244672->73248767] (req 73244672->73248767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.845298:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef0ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73244672->73248767] (req 73244672->73248767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.858336:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503675392 00000100:00100000:3.0:1789760765.858345:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503675392 00000100:00100000:3.0:1789760765.858425:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695956699497 time 695969788363 state 0x0 ref 1 00000100:00100000:3.0:1789760765.858440:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695970048779 time 695969788363 state 0x1 ref 1 00000100:00100000:3.0:1789760765.858466:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695970048779 time 695983073150 state 0x0 ref 1 00000100:00100000:3.0:1789760765.858477:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0774000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503675392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.858519:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73248768, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x22ef0 00010000:00010000:3.0:1789760765.858528:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.858543:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760765.858547:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.858553:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.858567:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.864752:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73248768 -> 73252864 00002000:00010000:3.0:1789760765.864758:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143088 -> 143096 00010000:00010000:3.0:1789760765.864770:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.864784:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.864792:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.864799:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.864805:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.864814:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.864835:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.864843:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef106 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73248768->73252863] (req 73248768->73252863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.864952:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0774000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503675392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6470us (6700us total) trans 4294985190 rc 0/0 00000100:00100000:3.0:1789760765.864962:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17894 00000100:00100000:3.0:1789760765.864972:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 695970048779 time 695983073150 state 0x2 ref 0 00000001:00080000:3.0:1789760765.898275:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985193 is committed 00000100:00100000:3.0:1789760765.904087:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760765.908654:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985194 is committed 00000100:00100000:3.0:1789760765.911845:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760765.924810:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73269248 -> 73273344 00002000:00010000:3.0:1789760765.924816:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143128 -> 143136 00010000:00010000:3.0:1789760765.924829:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef129 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73269248->73273343] (req 73269248->73273343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.924841:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef129 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73269248->73273343] (req 73269248->73273343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.924849:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef129 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73269248->73273343] (req 73269248->73273343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.924856:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760765.924860:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef129 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73269248->73273343] (req 73269248->73273343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.924869:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef129 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73269248->73273343] (req 73269248->73273343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.924888:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef129 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73269248->73273343] (req 73269248->73273343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.924895:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef129 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73269248->73273343] (req 73269248->73273343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.924992:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503676032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12921us (13151us total) trans 4294985195 rc 0/0 00000100:00100000:3.0:1789760765.925002:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17899 00000100:00100000:3.0:1789760765.925011:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696029227964 time 696036662342 state 0x2 ref 0 00000001:00080000:3.0:1789760765.960703:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985197 is committed 00000100:00100000:3.0:1789760765.965828:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760765.976499:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985198 is committed 00000100:00100000:3.0:1789760765.985706:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503676544 00000100:00100000:3.0:1789760765.985716:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503676544 00000100:00100000:3.0:1789760765.985779:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696078701212 time 696090746211 state 0x0 ref 1 00000100:00100000:3.0:1789760765.985795:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696091006627 time 696090746211 state 0x1 ref 1 00000100:00100000:3.0:1789760765.985828:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696091006627 time 696110434363 state 0x0 ref 1 00000100:00100000:3.0:1789760765.985841:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0777800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503676544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760765.985886:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73285632, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x22f38 00010000:00010000:3.0:1789760765.985895:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef145 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.985910:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760765.985915:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.985922:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73285632->73289727] (req 73285632->73289727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.985941:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73285632->73289727] (req 73285632->73289727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.994879:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985199 is committed 00000100:00080000:3.0:1789760766.004829:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760766.010573:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985200 is committed 00000100:00100000:3.0:1789760766.014324:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760766.020429:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985201 is committed 00000100:00100000:3.0:1789760766.028725:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503676928 00000100:00100000:3.0:1789760766.028736:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503676928 00000100:00100000:3.0:1789760766.028791:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696129755623 time 696139250227 state 0x0 ref 1 00000100:00100000:3.0:1789760766.028805:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696139510643 time 696139250227 state 0x1 ref 1 00000100:00100000:3.0:1789760766.028831:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696139510643 time 696153437840 state 0x0 ref 1 00000100:00100000:3.0:1789760766.028843:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0775f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503676928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760766.028887:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73297920, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x22f50 00010000:00010000:3.0:1789760766.028895:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef15a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.028910:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760766.028914:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.028920:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef15a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73297920->73302015] (req 73297920->73302015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.028937:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef15a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73297920->73302015] (req 73297920->73302015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.161875:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985210 is committed 00002000:00010000:3.0:1789760766.186182:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73334784 -> 73338880 00002000:00010000:3.0:1789760766.186193:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143256 -> 143264 00010000:00010000:3.0:1789760766.186207:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef199 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73334784->73338879] (req 73334784->73338879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186220:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef199 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73334784->73338879] (req 73334784->73338879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186229:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef199 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73334784->73338879] (req 73334784->73338879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186236:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760766.186240:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef199 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73334784->73338879] (req 73334784->73338879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186249:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef199 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73334784->73338879] (req 73334784->73338879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186269:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef199 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73334784->73338879] (req 73334784->73338879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186282:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef199 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73334784->73338879] (req 73334784->73338879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.186380:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503678080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16045us (16235us total) trans 4294985211 rc 0/0 00000100:00100000:3.0:1789760766.186700:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17915 00000100:00100000:3.0:1789760766.186711:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696279796072 time 696294926936 state 0x2 ref 0 00000100:00100000:3.0:1789760766.204428:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760766.251662:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985214 is committed 00000100:00100000:3.0:1789760766.258020:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760766.263598:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985215 is committed 00000001:00080000:3.0:1789760766.279611:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985216 is committed 00000100:00100000:3.0:1789760766.285284:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760766.289921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985217 is committed 00000100:00100000:3.0:1789760766.325770:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.353530:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.375934:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.386341:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00010000:00010000:0.0:1789760766.393800:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.393819:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.399070:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73392128 -> 73396224 00002000:00010000:0.0:1789760766.399075:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143368 -> 143376 00010000:00010000:0.0:1789760766.399087:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.399099:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.399109:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.399116:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760766.399120:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.399129:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.399149:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.399162:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef1fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73392128->73396223] (req 73392128->73396223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.399254:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503679872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5535us (5773us total) trans 4294985225 rc 0/0 00000100:00100000:0.0:1789760766.399265:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17929 00000100:00100000:0.0:1789760766.399274:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696511577381 time 696518310200 state 0x2 ref 0 00000100:00100000:3.0:1789760766.402519:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.406975:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985226 is committed 00000001:00080000:0.0:1789760766.415890:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985227 is committed 00000100:00100000:3.0:1789760766.422214:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503680256 00000100:00100000:3.0:1789760766.422225:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503680256 00000100:00100000:3.0:1789760766.422279:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696527649679 time 696535415457 state 0x0 ref 1 00000100:00100000:3.0:1789760766.422294:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696535675873 time 696535415457 state 0x1 ref 1 00000100:00100000:3.0:1789760766.422326:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696535675873 time 696546933531 state 0x0 ref 1 00000100:00100000:3.0:1789760766.422339:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503680256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760766.422478:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73404416, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x23020 00010000:00010000:3.0:1789760766.422488:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.422503:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760766.422508:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.422514:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.422529:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.430914:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73404416 -> 73408512 00002000:00010000:3.0:1789760766.430919:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143392 -> 143400 00010000:00010000:3.0:1789760766.430943:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.430954:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.430964:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.430977:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760766.430982:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.430991:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.431010:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.431017:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef210 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73404416->73408511] (req 73404416->73408511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.431117:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503680256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8777us (9536us total) trans 4294985228 rc 0/0 00000100:00100000:3.0:1789760766.431127:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17932 00000100:00100000:3.0:1789760766.431136:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696535675873 time 696546933531 state 0x2 ref 0 00000100:00100000:0.0:1789760766.436155:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.442155:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985229 is committed 00000100:00100000:0.0:1789760766.454225:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.461629:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985230 is committed 00000100:00100000:0.0:1789760766.468092:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.474125:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985231 is committed 00000100:00100000:0.0:1789760766.479011:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.479380:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503680768 00000100:00100000:3.0:1789760766.479987:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503680768 00000100:00100000:3.0:1789760766.480047:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696579462091 time 696592885879 state 0x0 ref 1 00000100:00100000:3.0:1789760766.480061:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696593146295 time 696592885879 state 0x1 ref 1 00000100:00100000:3.0:1789760766.480131:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696593146295 time 696604736444 state 0x0 ref 1 00000100:00100000:3.0:1789760766.480146:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503680768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760766.480189:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73420800, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x23040 00010000:00010000:3.0:1789760766.480198:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.480213:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760766.480218:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.480224:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.480237:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.482539:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760766.485222:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985232 is committed 00002000:00010000:3.0:1789760766.485335:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73420800 -> 73424896 00002000:00010000:3.0:1789760766.485340:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143424 -> 143432 00010000:00010000:3.0:1789760766.485353:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.485366:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.485375:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.485383:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760766.485419:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.485429:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.485448:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.485455:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef22c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73420800->73424895] (req 73420800->73424895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.485544:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503680768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5395us (6538us total) trans 4294985232 rc 0/0 00000100:00100000:3.0:1789760766.485554:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17936 00000100:00100000:3.0:1789760766.485563:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696593146295 time 696604736444 state 0x2 ref 0 00000100:00100000:0.0:1789760766.493270:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.498463:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985233 is committed 00000100:00100000:0.0:1789760766.507247:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.519345:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985234 is committed 00000100:00100000:0.0:1789760766.525280:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.534913:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985235 is committed 00000100:00100000:0.0:1789760766.539662:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.539821:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503681280 00000100:00100000:3.0:1789760766.539830:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503681280 00000100:00100000:3.0:1789760766.539882:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696633114599 time 696650758403 state 0x0 ref 1 00000100:00100000:3.0:1789760766.539898:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696651018819 time 696650758403 state 0x1 ref 1 00000100:00100000:3.0:1789760766.539969:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696651018819 time 696664574033 state 0x0 ref 1 00000100:00100000:3.0:1789760766.539982:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503681280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760766.540039:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73437184, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x23060 00010000:00010000:3.0:1789760766.540048:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.540063:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760766.540068:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.540074:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.540088:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.546578:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73437184 -> 73441280 00002000:00010000:3.0:1789760766.546584:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143456 -> 143464 00010000:00010000:3.0:1789760766.546662:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.546676:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.546695:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.546702:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760766.546706:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.546716:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.546906:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.546916:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef248 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73437184->73441279] (req 73437184->73441279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.547116:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503681280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7131us (7521us total) trans 4294985236 rc 0/0 00000100:00100000:3.0:1789760766.547127:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17940 00000100:00100000:3.0:1789760766.547137:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696651018819 time 696664574033 state 0x2 ref 0 00000100:00100000:0.0:1789760766.553484:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.559705:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985237 is committed 00000100:00100000:0.0:1789760766.563943:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.568990:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985238 is committed 00000100:00100000:0.0:1789760766.574430:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.579506:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985239 is committed 00000100:00100000:0.0:1789760766.582450:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.582695:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503681792 00000100:00100000:3.0:1789760766.582704:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503681792 00000100:00100000:3.0:1789760766.582765:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696689110287 time 696699352177 state 0x0 ref 1 00000100:00100000:3.0:1789760766.582781:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696699612593 time 696699352177 state 0x1 ref 1 00000100:00100000:3.0:1789760766.582854:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696699612593 time 696707459854 state 0x0 ref 1 00000100:00100000:3.0:1789760766.582866:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503681792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760766.582911:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73453568, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x23080 00010000:00010000:3.0:1789760766.582921:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.582935:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760766.582939:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.582945:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.582957:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.587524:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985240 is committed 00002000:00010000:3.0:1789760766.587676:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73453568 -> 73457664 00002000:00010000:3.0:1789760766.587693:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143488 -> 143496 00010000:00010000:3.0:1789760766.587705:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.587785:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.587797:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.587804:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760766.587809:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.587819:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.587840:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.587853:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef264 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73453568->73457663] (req 73453568->73457663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.587993:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503681792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5124us (5549us total) trans 4294985240 rc 0/0 00000100:00100000:3.0:1789760766.588004:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17944 00000100:00100000:3.0:1789760766.588014:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696699612593 time 696707459854 state 0x2 ref 0 00000100:00100000:0.0:1789760766.591381:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760766.600962:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73457664 -> 73461760 00002000:00010000:0.0:1789760766.600969:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143496 -> 143504 00010000:00010000:0.0:1789760766.600982:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef26b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73457664->73461759] (req 73457664->73461759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.600994:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef26b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73457664->73461759] (req 73457664->73461759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.601002:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef26b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73457664->73461759] (req 73457664->73461759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.601010:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760766.601014:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef26b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73457664->73461759] (req 73457664->73461759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.601022:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef26b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73457664->73461759] (req 73457664->73461759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.601095:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef26b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73457664->73461759] (req 73457664->73461759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.601106:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef26b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73457664->73461759] (req 73457664->73461759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.601219:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503681920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9291us (9842us total) trans 4294985241 rc 0/0 00000100:00100000:0.0:1789760766.601228:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17945 00000100:00100000:0.0:1789760766.601237:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696707720270 time 696716517604 state 0x2 ref 0 00000100:00100000:0.0:1789760766.606048:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.613538:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985242 is committed 00000100:00100000:0.0:1789760766.634176:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.634270:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503682304 00000100:00100000:3.0:1789760766.634281:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503682304 00000100:00100000:3.0:1789760766.634334:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696731115531 time 696742553176 state 0x0 ref 1 00000100:00100000:3.0:1789760766.634349:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696742813592 time 696742553176 state 0x1 ref 1 00000100:00100000:3.0:1789760766.634384:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696742813592 time 696758990037 state 0x0 ref 1 00000100:00100000:3.0:1789760766.634867:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503682304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760766.634918:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73469952, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x230a0 00010000:00010000:3.0:1789760766.634927:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef280 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.634943:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760766.634948:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.634955:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73469952->73474047] (req 73469952->73474047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.634974:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73469952->73474047] (req 73469952->73474047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.642350:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985244 is committed 00000100:00100000:0.0:1789760766.647067:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760766.652346:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985245 is committed 00000100:00100000:0.0:1789760766.660550:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760766.665865:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985246 is committed 00000100:00100000:0.0:1789760766.670807:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760766.683178:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760766.683270:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503682816 00000100:00100000:3.0:1789760766.683281:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503682816 00000100:00100000:3.0:1789760766.683332:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696785796757 time 696796593923 state 0x0 ref 1 00000100:00100000:3.0:1789760766.683346:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696796854339 time 696796593923 state 0x1 ref 1 00000100:00100000:3.0:1789760766.683376:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696796854339 time 696807982891 state 0x0 ref 1 00000100:00100000:3.0:1789760766.683546:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503682816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760766.683665:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73486336, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x230c0 00010000:00010000:3.0:1789760766.683675:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.683699:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760766.683704:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.683710:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.683786:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.688068:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985248 is committed 00002000:00010000:3.0:1789760766.688172:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73486336 -> 73490432 00002000:00010000:3.0:1789760766.688178:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143552 -> 143560 00010000:00010000:3.0:1789760766.688191:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.688204:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.688214:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.688221:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760766.688276:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.688287:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.688307:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.688315:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef29c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73486336->73490431] (req 73486336->73490431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.688543:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503682816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5150us (5367us total) trans 4294985248 rc 0/0 00000100:00100000:3.0:1789760766.688554:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17952 00000100:00100000:3.0:1789760766.688563:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696796854339 time 696807982891 state 0x2 ref 0 00000100:00100000:0.0:1789760766.692547:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.706061:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985249 is committed 00000100:00100000:0.0:1789760766.714648:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760766.730333:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760766.748162:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.763823:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985253 is committed 00000100:00100000:0.0:1789760766.770298:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760766.778750:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73510912 -> 73515008 00002000:00010000:0.0:1789760766.778755:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143600 -> 143608 00010000:00010000:0.0:1789760766.778768:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ef2c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73510912->73515007] (req 73510912->73515007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.778781:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ef2c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73510912->73515007] (req 73510912->73515007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.778791:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ef2c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73510912->73515007] (req 73510912->73515007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.778799:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760766.778804:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ef2c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73510912->73515007] (req 73510912->73515007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.778812:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ef2c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73510912->73515007] (req 73510912->73515007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.778832:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ef2c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73510912->73515007] (req 73510912->73515007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.778840:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083ef2c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73510912->73515007] (req 73510912->73515007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.779156:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503683584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8431us (8861us total) trans 4294985254 rc 0/0 00000100:00100000:0.0:1789760766.779168:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17958 00000100:00100000:0.0:1789760766.779178:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696883380517 time 696895312445 state 0x2 ref 0 00000100:00100000:0.0:1789760766.784642:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760766.798485:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503683840 00000100:00100000:0.0:1789760766.798495:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503683840 00000100:00100000:0.0:1789760766.798547:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696895572861 time 696909902143 state 0x0 ref 1 00000100:00100000:0.0:1789760766.798566:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696910162559 time 696909902143 state 0x1 ref 1 00000100:00100000:0.0:1789760766.798593:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696910162559 time 696923200119 state 0x0 ref 1 00000100:00100000:0.0:1789760766.798605:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0775f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503683840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760766.798654:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73519104, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23100 00010000:00010000:0.0:1789760766.798664:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.798690:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760766.798695:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.798702:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.798716:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.805454:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73519104 -> 73523200 00002000:00010000:0.0:1789760766.805460:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143616 -> 143624 00010000:00010000:0.0:1789760766.805474:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.805487:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.805497:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.805505:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760766.805511:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.805520:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.805541:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.805549:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef2d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73519104->73523199] (req 73519104->73523199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.805657:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0775f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503683840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7046us (7678us total) trans 4294985256 rc 0/0 00000100:00100000:0.0:1789760766.805668:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17960 00000100:00100000:0.0:1789760766.805690:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696910162559 time 696923200119 state 0x2 ref 0 00000100:00100000:0.0:1789760766.809688:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760766.818246:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73523200 -> 73527296 00002000:00010000:0.0:1789760766.818257:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143624 -> 143632 00010000:00010000:0.0:1789760766.818271:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef2db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73523200->73527295] (req 73523200->73527295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.818339:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef2db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73523200->73527295] (req 73523200->73527295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.818350:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef2db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73523200->73527295] (req 73523200->73527295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.818357:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760766.818362:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef2db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73523200->73527295] (req 73523200->73527295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.818371:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef2db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73523200->73527295] (req 73523200->73527295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.818476:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef2db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73523200->73527295] (req 73523200->73527295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.818485:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef2db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73523200->73527295] (req 73523200->73527295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.818588:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503683968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8510us (8924us total) trans 4294985257 rc 0/0 00000100:00100000:0.0:1789760766.818598:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17961 00000100:00100000:0.0:1789760766.818608:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 696923460535 time 696934669203 state 0x2 ref 0 00000100:00100000:0.0:1789760766.828608:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503684096 00000100:00100000:0.0:1789760766.828625:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503684096 00000001:00080000:0.0:1789760766.837767:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985258 is committed 00000001:00080000:0.0:1789760766.862738:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985259 is committed 00000100:00100000:0.0:1789760766.869315:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760766.877696:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985260 is committed 00000100:00100000:0.0:1789760766.888647:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760766.926586:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760766.958195:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73551872 -> 73555968 00002000:00010000:0.0:1789760766.958201:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143680 -> 143688 00010000:00010000:0.0:1789760766.958213:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083ef30c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73551872->73555967] (req 73551872->73555967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958225:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083ef30c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73551872->73555967] (req 73551872->73555967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958233:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083ef30c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73551872->73555967] (req 73551872->73555967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958239:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760766.958244:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083ef30c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73551872->73555967] (req 73551872->73555967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958252:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083ef30c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73551872->73555967] (req 73551872->73555967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958270:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083ef30c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73551872->73555967] (req 73551872->73555967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.958277:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083ef30c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73551872->73555967] (req 73551872->73555967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.958370:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503684864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8313us (8876us total) trans 4294985264 rc 0/0 00000100:00100000:0.0:1789760766.958381:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17968 00000100:00100000:0.0:1789760766.958424:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697051786805 time 697074646583 state 0x2 ref 0 00000100:00100000:0.0:1789760766.962165:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503684992 00000100:00100000:0.0:1789760766.962174:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503684992 00000100:00100000:0.0:1789760766.962226:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697051786805 time 697074646583 state 0x0 ref 1 00000100:00100000:0.0:1789760766.962241:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697074906999 time 697074646583 state 0x1 ref 1 00000100:00100000:0.0:1789760766.962316:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697074906999 time 697086921044 state 0x0 ref 1 00000100:00100000:0.0:1789760766.962328:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503684992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760766.962372:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73555968, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23148 00010000:00010000:0.0:1789760766.962381:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.962432:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760766.962437:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.962442:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.962456:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.968306:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73555968 -> 73560064 00002000:00010000:0.0:1789760766.968312:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143688 -> 143696 00010000:00010000:0.0:1789760766.968325:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.968336:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.968345:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.968352:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760766.970338:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.970423:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.970453:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.970464:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef313 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73555968->73560063] (req 73555968->73560063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.970577:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503684992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8246us (8460us total) trans 4294985265 rc 0/0 00000100:00100000:0.0:1789760766.970588:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17969 00000100:00100000:0.0:1789760766.970598:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697074906999 time 697086921044 state 0x2 ref 0 00000100:00100000:3.0:1789760766.973974:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760766.999314:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503685376 00000100:00100000:0.0:1789760766.999325:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503685376 00000100:00100000:0.0:1789760766.999526:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697099115597 time 697110592132 state 0x0 ref 1 00000100:00100000:0.0:1789760766.999543:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697110852548 time 697110592132 state 0x1 ref 1 00000100:00100000:0.0:1789760766.999571:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697110852548 time 697124177532 state 0x0 ref 1 00000100:00100000:0.0:1789760766.999582:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503685376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760766.999632:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73568256, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23160 00010000:00010000:0.0:1789760766.999641:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.999656:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760766.999661:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.999667:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.999690:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.009529:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73568256 -> 73572352 00002000:00010000:0.0:1789760767.009535:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143712 -> 143720 00010000:00010000:0.0:1789760767.009548:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.009564:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.009573:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.009580:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.009585:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.009593:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.009612:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.009620:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef328 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73568256->73572351] (req 73568256->73572351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.009728:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503685376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10144us (10741us total) trans 4294985268 rc 0/0 00000100:00100000:0.0:1789760767.009738:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17972 00000100:00100000:0.0:1789760767.009748:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697110852548 time 697124177532 state 0x2 ref 0 00000100:00100000:3.0:1789760767.015252:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760767.028432:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73572352 -> 73576448 00002000:00010000:3.0:1789760767.028443:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143720 -> 143728 00010000:00010000:3.0:1789760767.028457:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef32f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73572352->73576447] (req 73572352->73576447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.028469:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef32f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73572352->73576447] (req 73572352->73576447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.028477:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef32f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73572352->73576447] (req 73572352->73576447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.028484:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.028489:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef32f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73572352->73576447] (req 73572352->73576447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.028498:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef32f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73572352->73576447] (req 73572352->73576447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.028517:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef32f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73572352->73576447] (req 73572352->73576447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.028529:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef32f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73572352->73576447] (req 73572352->73576447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.028628:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503685504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10666us (13381us total) trans 4294985269 rc 0/0 00000100:00100000:3.0:1789760767.028638:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17973 00000100:00100000:3.0:1789760767.028648:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697124437948 time 697142550060 state 0x2 ref 0 00000001:00080000:3.0:1789760767.062495:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985271 is committed 00000100:00100000:3.0:1789760767.069724:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760767.070890:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503685888 00000100:00100000:0.0:1789760767.070905:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503685888 00000100:00100000:0.0:1789760767.070969:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697158190669 time 697181165371 state 0x0 ref 1 00000100:00100000:0.0:1789760767.070985:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697181425787 time 697181165371 state 0x1 ref 1 00000100:00100000:0.0:1789760767.071020:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697181425787 time 697195626399 state 0x0 ref 1 00000100:00100000:0.0:1789760767.071033:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503685888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.071075:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73584640, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23180 00010000:00010000:0.0:1789760767.071084:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.071100:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.071105:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.071111:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.071124:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.076031:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985272 is committed 00002000:00010000:0.0:1789760767.076154:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73584640 -> 73588736 00002000:00010000:0.0:1789760767.076160:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143744 -> 143752 00010000:00010000:0.0:1789760767.076173:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.076186:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.076200:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.076206:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.076212:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.076219:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.076237:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.076243:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef344 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73584640->73588735] (req 73584640->73588735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.076335:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503685888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5299us (6614us total) trans 4294985272 rc 0/0 00000100:00100000:0.0:1789760767.076345:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17976 00000100:00100000:0.0:1789760767.076355:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697181425787 time 697195626399 state 0x2 ref 0 00000100:00100000:3.0:1789760767.080527:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503686016 00000100:00100000:3.0:1789760767.080539:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503686016 00000100:00100000:3.0:1789760767.080651:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697181425787 time 697195626399 state 0x0 ref 1 00000100:00100000:3.0:1789760767.080664:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697195886815 time 697195626399 state 0x1 ref 1 00000100:00100000:3.0:1789760767.080817:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697195886815 time 697205422806 state 0x0 ref 1 00000100:00100000:3.0:1789760767.080831:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503686016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.080895:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73588736, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23188 00010000:00010000:3.0:1789760767.080905:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.080921:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.080927:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.080933:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.080948:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.086722:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73588736 -> 73592832 00002000:00010000:3.0:1789760767.086728:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143752 -> 143760 00010000:00010000:3.0:1789760767.086741:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.086753:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.086762:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.086770:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.086775:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.086784:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.086803:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.086816:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef34b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73588736->73592831] (req 73588736->73592831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.087936:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503686016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7100us (7652us total) trans 4294985273 rc 0/0 00000100:00100000:3.0:1789760767.087950:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17977 00000100:00100000:3.0:1789760767.087960:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697195886815 time 697205422806 state 0x2 ref 0 00000100:00100000:3.0:1789760767.094854:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503686144 00000100:00100000:3.0:1789760767.094865:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503686144 00000100:00100000:3.0:1789760767.094919:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697195886815 time 697205422806 state 0x0 ref 1 00000100:00100000:3.0:1789760767.094934:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697205683222 time 697205422806 state 0x1 ref 1 00000100:00100000:3.0:1789760767.094971:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697205683222 time 697219577197 state 0x0 ref 1 00000100:00100000:3.0:1789760767.094992:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503686144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.095035:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73592832, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23190 00010000:00010000:3.0:1789760767.095043:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.095058:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.095062:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.095068:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.095087:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.099924:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73592832 -> 73596928 00002000:00010000:3.0:1789760767.099930:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143760 -> 143768 00010000:00010000:3.0:1789760767.099943:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.099956:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.099964:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.099971:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.099975:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.099983:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.100002:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.100008:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef352 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73592832->73596927] (req 73592832->73596927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.100238:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503686144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5233us (5523us total) trans 4294985274 rc 0/0 00000100:00100000:3.0:1789760767.100251:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17978 00000100:00100000:3.0:1789760767.100262:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697205683222 time 697219577197 state 0x2 ref 0 00000001:00080000:3.0:1789760767.129067:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985276 is committed 00000100:00100000:3.0:1789760767.134139:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760767.134387:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503686528 00000100:00100000:0.0:1789760767.134426:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503686528 00000100:00100000:0.0:1789760767.134486:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697233658723 time 697249324664 state 0x0 ref 1 00000100:00100000:0.0:1789760767.134501:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697249585080 time 697249324664 state 0x1 ref 1 00000100:00100000:0.0:1789760767.134535:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697249585080 time 697259141329 state 0x0 ref 1 00000100:00100000:0.0:1789760767.134547:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503686528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.134592:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73605120, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x231a8 00010000:00010000:0.0:1789760767.134602:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.134617:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.134622:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.134628:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.134641:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.139594:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985277 is committed 00002000:00010000:0.0:1789760767.139880:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73605120 -> 73609216 00002000:00010000:0.0:1789760767.139885:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143784 -> 143792 00010000:00010000:0.0:1789760767.139902:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.139917:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.139929:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.139945:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.139950:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.139963:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.139987:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.139997:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef367 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73605120->73609215] (req 73605120->73609215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.140107:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503686528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5556us (5972us total) trans 4294985277 rc 0/0 00000100:00100000:0.0:1789760767.140120:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17981 00000100:00100000:0.0:1789760767.140131:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697249585080 time 697259141329 state 0x2 ref 0 00000100:00100000:3.0:1789760767.143530:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760767.151770:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985278 is committed 00000001:00080000:3.0:1789760767.165477:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985279 is committed 00000100:00100000:3.0:1789760767.170232:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760767.175723:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985280 is committed 00000100:00100000:0.0:1789760767.179088:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.179174:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503687040 00000100:00100000:3.0:1789760767.179185:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503687040 00000100:00100000:3.0:1789760767.179241:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697283679087 time 697295095863 state 0x0 ref 1 00000100:00100000:3.0:1789760767.179256:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697295356279 time 697295095863 state 0x1 ref 1 00000100:00100000:3.0:1789760767.179282:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697295356279 time 697303888286 state 0x0 ref 1 00000100:00100000:3.0:1789760767.179294:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503687040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.179339:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73621504, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x231c8 00010000:00010000:3.0:1789760767.179349:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.179370:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.179374:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.179380:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.179440:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.185265:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73621504 -> 73625600 00002000:00010000:3.0:1789760767.185273:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143816 -> 143824 00010000:00010000:3.0:1789760767.185287:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.185300:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.185314:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.185322:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.185326:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.185335:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.185567:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.185577:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef383 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73621504->73625599] (req 73621504->73625599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.185732:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503687040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6434us (6647us total) trans 4294985281 rc 0/0 00000100:00100000:3.0:1789760767.185743:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17985 00000100:00100000:3.0:1789760767.185753:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697295356279 time 697303888286 state 0x2 ref 0 00000100:00100000:3.0:1789760767.190131:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760767.213908:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503687424 00000100:00100000:0.0:1789760767.213918:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503687424 00000100:00100000:0.0:1789760767.213970:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697315193500 time 697327881497 state 0x0 ref 1 00000100:00100000:0.0:1789760767.213983:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697328141913 time 697327881497 state 0x1 ref 1 00000100:00100000:0.0:1789760767.214013:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697328141913 time 697338620224 state 0x0 ref 1 00000100:00100000:0.0:1789760767.214025:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503687424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.214068:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73633792, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x231e0 00010000:00010000:0.0:1789760767.214076:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.214090:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.214094:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.214100:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.214112:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.225151:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73633792 -> 73637888 00002000:00010000:0.0:1789760767.225156:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143840 -> 143848 00010000:00010000:0.0:1789760767.225169:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.225181:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.225190:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.225197:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.225203:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.225212:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.225231:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.225239:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef398 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73633792->73637887] (req 73633792->73637887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.225384:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503687424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11354us (11576us total) trans 4294985284 rc 0/0 00000100:00100000:0.0:1789760767.225440:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17988 00000100:00100000:0.0:1789760767.225449:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697328141913 time 697338620224 state 0x2 ref 0 00000100:00100000:3.0:1789760767.236440:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503687552 00000100:00100000:3.0:1789760767.236452:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503687552 00000100:00100000:3.0:1789760767.236511:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697328141913 time 697338620224 state 0x0 ref 1 00000100:00100000:3.0:1789760767.236527:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697338880640 time 697338620224 state 0x1 ref 1 00000100:00100000:3.0:1789760767.236551:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697338880640 time 697361158114 state 0x0 ref 1 00000100:00100000:3.0:1789760767.236563:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503687552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.236611:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73637888, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x231e8 00010000:00010000:3.0:1789760767.236620:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.236640:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.236645:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.236651:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.236664:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.243593:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73637888 -> 73641984 00002000:00010000:3.0:1789760767.243599:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143848 -> 143856 00010000:00010000:3.0:1789760767.243611:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.243622:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.243631:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.243637:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.243642:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.243650:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.243672:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.243691:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef39f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73637888->73641983] (req 73637888->73641983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.243893:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503687552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7327us (8115us total) trans 4294985285 rc 0/0 00000100:00100000:3.0:1789760767.243904:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17989 00000100:00100000:3.0:1789760767.243913:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697338880640 time 697361158114 state 0x2 ref 0 00002000:00010000:3.0:1789760767.253858:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73641984 -> 73646080 00002000:00010000:3.0:1789760767.253863:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143856 -> 143864 00010000:00010000:3.0:1789760767.253876:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef3a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73641984->73646079] (req 73641984->73646079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.253889:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef3a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73641984->73646079] (req 73641984->73646079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.253897:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef3a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73641984->73646079] (req 73641984->73646079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.253904:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.253908:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef3a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73641984->73646079] (req 73641984->73646079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.253921:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef3a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73641984->73646079] (req 73641984->73646079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.253942:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef3a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73641984->73646079] (req 73641984->73646079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.253950:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef3a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73641984->73646079] (req 73641984->73646079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.254060:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503687680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6161us (7251us total) trans 4294985286 rc 0/0 00000100:00100000:3.0:1789760767.254070:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17990 00000100:00100000:3.0:1789760767.254080:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697361418530 time 697372489778 state 0x2 ref 0 00000100:00100000:0.0:1789760767.270332:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760767.277686:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985288 is committed 00000100:00100000:3.0:1789760767.280999:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503688064 00000100:00100000:3.0:1789760767.281009:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503688064 00000100:00100000:3.0:1789760767.281066:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697383039068 time 697395280058 state 0x0 ref 1 00000100:00100000:3.0:1789760767.281081:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697395540474 time 697395280058 state 0x1 ref 1 00000100:00100000:3.0:1789760767.281110:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697395540474 time 697405717471 state 0x0 ref 1 00000100:00100000:3.0:1789760767.281122:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503688064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.281169:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73654272, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23208 00010000:00010000:3.0:1789760767.281178:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.281195:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.281199:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.281205:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.281218:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.286316:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73654272 -> 73658368 00002000:00010000:0.0:1789760767.286321:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143880 -> 143888 00010000:00010000:0.0:1789760767.286334:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.286346:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.286360:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.286367:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.286372:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.286381:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.286526:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.286536:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef3bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73654272->73658367] (req 73654272->73658367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.286663:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503688064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5538us (5746us total) trans 4294985289 rc 0/0 00000100:00100000:0.0:1789760767.286674:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17993 00000100:00100000:0.0:1789760767.286694:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697395540474 time 697405717471 state 0x2 ref 0 00000100:00100000:3.0:1789760767.291643:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503688192 00000100:00100000:3.0:1789760767.291654:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503688192 00000100:00100000:3.0:1789760767.291780:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697395540474 time 697405717471 state 0x0 ref 1 00000100:00100000:3.0:1789760767.291795:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697405977887 time 697405717471 state 0x1 ref 1 00000100:00100000:3.0:1789760767.291865:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697405977887 time 697416470729 state 0x0 ref 1 00000100:00100000:3.0:1789760767.291878:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503688192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.291930:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73658368, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23210 00010000:00010000:3.0:1789760767.291941:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.291956:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.291961:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.291967:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.291985:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.301029:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985290 is committed 00002000:00010000:3.0:1789760767.301210:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73658368 -> 73662464 00002000:00010000:3.0:1789760767.301216:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143888 -> 143896 00010000:00010000:3.0:1789760767.301228:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.301311:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.301319:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.301328:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.301332:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.301341:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.301361:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.301368:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef3c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73658368->73662463] (req 73658368->73662463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.301640:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503688192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9761us (10199us total) trans 4294985290 rc 0/0 00000100:00100000:3.0:1789760767.301651:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17994 00000100:00100000:3.0:1789760767.301661:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697405977887 time 697416470729 state 0x2 ref 0 00000001:00080000:0.0:1789760767.311701:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985291 is committed 00000100:00100000:0.0:1789760767.318311:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503688448 00000100:00100000:0.0:1789760767.318321:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503688448 00000100:00100000:0.0:1789760767.318372:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697416731145 time 697430341237 state 0x0 ref 1 00000100:00100000:0.0:1789760767.318428:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697430601653 time 697430341237 state 0x1 ref 1 00000100:00100000:0.0:1789760767.318457:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697430601653 time 697443063644 state 0x0 ref 1 00000100:00100000:0.0:1789760767.318469:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503688448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.318512:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73666560, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23220 00010000:00010000:0.0:1789760767.318522:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.318538:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.318542:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.318547:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.318560:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.323991:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73666560 -> 73670656 00002000:00010000:0.0:1789760767.323997:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143904 -> 143912 00010000:00010000:0.0:1789760767.324010:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324023:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324032:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324113:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.324118:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324127:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324148:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.324156:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef3d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73666560->73670655] (req 73666560->73670655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.324283:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503688448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5812us (6326us total) trans 4294985292 rc 0/0 00000100:00100000:0.0:1789760767.324294:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17996 00000100:00100000:0.0:1789760767.324304:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697430601653 time 697443063644 state 0x2 ref 0 00000100:00100000:3.0:1789760767.328654:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760767.328785:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503688576 00000100:00100000:0.0:1789760767.328796:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503688576 00000100:00100000:0.0:1789760767.328854:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697430601653 time 697443063644 state 0x0 ref 1 00000100:00100000:0.0:1789760767.328871:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697443324060 time 697443063644 state 0x1 ref 1 00000100:00100000:0.0:1789760767.328902:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697443324060 time 697453509107 state 0x0 ref 1 00000100:00100000:0.0:1789760767.328918:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503688576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.328976:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73670656, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23228 00010000:00010000:0.0:1789760767.328987:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef3d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.329004:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.329008:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.329014:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef3d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73670656->73674751] (req 73670656->73674751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.329027:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef3d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73670656->73674751] (req 73670656->73674751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.335715:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985293 is committed 00000100:00100000:3.0:1789760767.342110:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503688704 00000100:00100000:3.0:1789760767.342120:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503688704 00000100:00100000:3.0:1789760767.342490:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697443324060 time 697453509107 state 0x0 ref 1 00000100:00100000:3.0:1789760767.342504:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697453769523 time 697453509107 state 0x1 ref 1 00000100:00100000:3.0:1789760767.342530:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697453769523 time 697467136923 state 0x0 ref 1 00000100:00100000:3.0:1789760767.342544:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503688704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.342646:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73674752, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23230 00010000:00010000:3.0:1789760767.342655:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.342671:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.342686:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.342692:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.342706:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.350893:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73674752 -> 73678848 00002000:00010000:3.0:1789760767.351059:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 143920 -> 143928 00010000:00010000:3.0:1789760767.351075:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.351088:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.351096:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.351103:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.351108:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.351116:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.351136:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.351144:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef3de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73674752->73678847] (req 73674752->73678847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.351666:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503688704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8689us (9202us total) trans 4294985294 rc 0/0 00000100:00100000:3.0:1789760767.351692:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 17998 00000100:00100000:3.0:1789760767.351703:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697453769523 time 697467136923 state 0x2 ref 0 00000001:00080000:0.0:1789760767.364487:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985295 is committed 00000100:00100000:0.0:1789760767.375280:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503688960 00000100:00100000:0.0:1789760767.375290:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503688960 00000100:00100000:0.0:1789760767.375340:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697467397339 time 697480906991 state 0x0 ref 1 00000100:00100000:0.0:1789760767.375355:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697481167407 time 697480906991 state 0x1 ref 1 00000100:00100000:0.0:1789760767.375384:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697481167407 time 697499991406 state 0x0 ref 1 00000100:00100000:0.0:1789760767.392859:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.429137:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.429180:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.429213:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.429238:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.429256:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.429274:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.429299:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503689600 00000100:00100000:3.0:1789760767.429308:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503689600 00000100:00100000:3.0:1789760767.429358:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503689856 00000100:00100000:3.0:1789760767.429361:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503689856 00000100:00100000:3.0:1789760767.429374:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503689728 00000100:00100000:3.0:1789760767.429376:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503689728 00000100:00100000:3.0:1789760767.429429:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503689984 00000100:00100000:3.0:1789760767.429433:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503689984 00000100:00100000:3.0:1789760767.429453:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 941 00000100:00100000:3.0:1789760767.429462:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2391:x1876699503689600:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760767.429693:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2391:x1876699503689600:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 229us (509us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760767.429710:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 941 00000001:00080000:3.0:1789760767.459568:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985300 is committed 00000100:00100000:3.0:1789760767.462910:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760767.471583:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985301 is committed 00000001:00080000:3.0:1789760767.485946:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985302 is committed 00000100:00100000:3.0:1789760767.504548:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00010000:00010000:3.0:1789760767.508015:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760767.521367:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.522150:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697615118209 time 697632874372 state 0x0 ref 1 00000100:00100000:3.0:1789760767.522167:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697633134788 time 697632874372 state 0x1 ref 1 00000100:00100000:3.0:1789760767.522201:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697633134788 time 697646807643 state 0x0 ref 1 00000100:00100000:3.0:1789760767.522212:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503690752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.522258:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73719808, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23288 00010000:00010000:3.0:1789760767.522266:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.522281:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.522286:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.522292:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.522304:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.528488:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985305 is committed 00002000:00010000:0.0:1789760767.528814:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73719808 -> 73723904 00002000:00010000:0.0:1789760767.528821:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144008 -> 144016 00010000:00010000:0.0:1789760767.528834:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.528847:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.528856:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.528863:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.528868:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.528878:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.528898:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.528905:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef42b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73719808->73723903] (req 73719808->73723903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.529066:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503690752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6850us (7704us total) trans 4294985305 rc 0/0 00000100:00100000:0.0:1789760767.529077:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18009 00000100:00100000:0.0:1789760767.529086:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697633134788 time 697646807643 state 0x2 ref 0 00000100:00100000:3.0:1789760767.533971:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760767.541658:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985306 is committed 00000001:00080000:3.0:1789760767.550949:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985307 is committed 00000001:00080000:3.0:1789760767.563047:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985308 is committed 00000100:00100000:0.0:1789760767.567090:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.567179:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503691264 00000100:00100000:3.0:1789760767.567189:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503691264 00000100:00100000:3.0:1789760767.567241:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697670783133 time 697680538227 state 0x0 ref 1 00000100:00100000:3.0:1789760767.567260:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697680798643 time 697680538227 state 0x1 ref 1 00000100:00100000:3.0:1789760767.567292:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697680798643 time 697691898504 state 0x0 ref 1 00000100:00100000:3.0:1789760767.567305:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503691264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.567352:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73736192, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x232a8 00010000:00010000:3.0:1789760767.567361:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef447 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567375:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.567380:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.567386:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73736192->73740287] (req 73736192->73740287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567444:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73736192->73740287] (req 73736192->73740287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.572968:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985309 is committed 00000100:00100000:3.0:1789760767.576803:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503691392 00000100:00100000:3.0:1789760767.576814:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503691392 00000100:00100000:3.0:1789760767.576868:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697680798643 time 697691898504 state 0x0 ref 1 00000100:00100000:3.0:1789760767.576883:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697692158920 time 697691898504 state 0x1 ref 1 00000100:00100000:3.0:1789760767.576949:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697692158920 time 697701554273 state 0x0 ref 1 00000100:00100000:3.0:1789760767.576963:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503691392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.577008:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73740288, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x232b0 00010000:00010000:3.0:1789760767.577018:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef44e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.577034:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.577038:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.577044:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef44e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73740288->73744383] (req 73740288->73744383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.577057:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef44e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73740288->73744383] (req 73740288->73744383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.582180:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985310 is committed 00000001:00080000:0.0:1789760767.591912:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985311 is committed 00000100:00100000:0.0:1789760767.595894:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760767.601150:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985312 is committed 00000100:00100000:3.0:1789760767.606385:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760767.612514:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985313 is committed 00000100:00100000:3.0:1789760767.619475:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503691904 00000100:00100000:3.0:1789760767.619487:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503691904 00000100:00100000:3.0:1789760767.619541:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697720990192 time 697731222182 state 0x0 ref 1 00000100:00100000:3.0:1789760767.619557:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697731482598 time 697731222182 state 0x1 ref 1 00000100:00100000:3.0:1789760767.619596:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697731482598 time 697744202210 state 0x0 ref 1 00000100:00100000:3.0:1789760767.619608:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503691904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.619652:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73756672, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x232d0 00010000:00010000:3.0:1789760767.619661:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.619689:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.619693:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.619699:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.619712:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.632486:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985314 is committed 00002000:00010000:3.0:1789760767.632664:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73756672 -> 73760768 00002000:00010000:3.0:1789760767.632669:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144080 -> 144088 00010000:00010000:3.0:1789760767.632689:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.632702:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.632711:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.632719:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.632724:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.632733:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.632759:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.632768:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef46a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73756672->73760767] (req 73756672->73760767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.632917:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503691904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13305us (13569us total) trans 4294985314 rc 0/0 00000100:00100000:3.0:1789760767.632928:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18018 00000100:00100000:3.0:1789760767.632937:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697731482598 time 697744202210 state 0x2 ref 0 00000100:00100000:0.0:1789760767.641786:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760767.654233:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985315 is committed 00000001:00080000:0.0:1789760767.673476:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985316 is committed 00000100:00100000:3.0:1789760767.687538:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503692288 00000100:00100000:3.0:1789760767.687554:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503692288 00000100:00100000:3.0:1789760767.687619:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697766903773 time 697790653922 state 0x0 ref 1 00000100:00100000:3.0:1789760767.687635:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697790914338 time 697790653922 state 0x1 ref 1 00000100:00100000:3.0:1789760767.687663:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697790914338 time 697812270446 state 0x0 ref 1 00000100:00100000:3.0:1789760767.687685:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffb6f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503692288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.687734:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73768960, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x232e8 00010000:00010000:3.0:1789760767.687743:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef47f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.687759:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.687764:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.687770:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef47f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73768960->73773055] (req 73768960->73773055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.687783:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef47f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73768960->73773055] (req 73768960->73773055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.697866:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985317 is committed 00000100:00100000:0.0:1789760767.702974:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760767.703051:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503692416 00000100:00100000:3.0:1789760767.703061:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503692416 00000100:00100000:3.0:1789760767.703112:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697790914338 time 697812270446 state 0x0 ref 1 00000100:00100000:3.0:1789760767.703127:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697812530862 time 697812270446 state 0x1 ref 1 00000100:00100000:3.0:1789760767.703160:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697812530862 time 697827766232 state 0x0 ref 1 00000100:00100000:3.0:1789760767.703172:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503692416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.703218:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73773056, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x232f0 00010000:00010000:3.0:1789760767.703239:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.703256:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760767.703260:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.703267:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.703282:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.710378:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73773056 -> 73777152 00002000:00010000:0.0:1789760767.710384:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144112 -> 144120 00010000:00010000:0.0:1789760767.710533:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.710548:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.710557:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.710565:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.710570:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.710587:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.710607:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.710616:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef486 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73773056->73777151] (req 73773056->73777151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.710820:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503692416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7636us (7841us total) trans 4294985318 rc 0/0 00000100:00100000:0.0:1789760767.710830:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18022 00000100:00100000:0.0:1789760767.710840:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697812530862 time 697827766232 state 0x2 ref 0 00000001:00080000:0.0:1789760767.723215:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985319 is committed 00000100:00100000:0.0:1789760767.738786:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760767.748694:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503692928 00000100:00100000:0.0:1789760767.748704:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503692928 00000100:00100000:0.0:1789760767.748768:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697854168214 time 697863745730 state 0x0 ref 1 00000100:00100000:0.0:1789760767.748784:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697864006146 time 697863745730 state 0x1 ref 1 00000100:00100000:0.0:1789760767.748814:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697864006146 time 697873420967 state 0x0 ref 1 00000100:00100000:0.0:1789760767.748827:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503692928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.748876:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73789440, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23310 00010000:00010000:0.0:1789760767.748885:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.748900:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.748905:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.748911:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.748923:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.754968:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73789440 -> 73793536 00002000:00010000:0.0:1789760767.754974:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144144 -> 144152 00010000:00010000:0.0:1789760767.754987:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.755000:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.755009:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.755017:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.755021:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.755036:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.755055:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.755063:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73789440->73793535] (req 73789440->73793535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.755193:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503692928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6363us (6593us total) trans 4294985322 rc 0/0 00000100:00100000:0.0:1789760767.755204:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18026 00000100:00100000:0.0:1789760767.755214:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697864006146 time 697873420967 state 0x2 ref 0 00000100:00100000:0.0:1789760767.769423:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697883731226 time 697894026453 state 0x0 ref 1 00000100:00100000:0.0:1789760767.769439:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503693184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.769485:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73797632, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x23320 00010000:00010000:0.0:1789760767.769494:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.769509:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.769514:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.769520:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.769535:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.777550:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73797632 -> 73801728 00002000:00010000:0.0:1789760767.777556:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144160 -> 144168 00010000:00010000:0.0:1789760767.777569:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.777583:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.777592:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.777611:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.777616:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.777625:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.777646:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.777654:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73797632->73801727] (req 73797632->73801727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.777769:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503693184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8329us (8739us total) trans 4294985324 rc 0/0 00000100:00100000:0.0:1789760767.777780:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18028 00000100:00100000:0.0:1789760767.777789:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697883731226 time 697894026453 state 0x2 ref 0 00000001:00080000:0.0:1789760767.794960:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985325 is committed 00000001:00080000:0.0:1789760767.812560:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985326 is committed 00000001:00080000:0.0:1789760767.826091:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985327 is committed 00000100:00100000:0.0:1789760767.837853:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503693696 00000100:00100000:0.0:1789760767.837863:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503693696 00000100:00100000:0.0:1789760767.837921:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697932506887 time 697943483594 state 0x0 ref 1 00000100:00100000:0.0:1789760767.837937:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697943744010 time 697943483594 state 0x1 ref 1 00000100:00100000:0.0:1789760767.838013:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697943744010 time 697962618423 state 0x0 ref 1 00000100:00100000:0.0:1789760767.838027:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503693696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.838076:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73814016, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23340 00010000:00010000:0.0:1789760767.838085:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.838101:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760767.838106:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.838112:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73814016->73818111] (req 73814016->73818111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.838145:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef4cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73814016->73818111] (req 73814016->73818111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.844892:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985328 is committed 00000001:00080000:0.0:1789760767.854326:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985329 is committed 00002000:00010000:3.0:1789760767.854618:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73818112 -> 73822208 00002000:00010000:3.0:1789760767.854624:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144200 -> 144208 00010000:00010000:3.0:1789760767.854637:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef4d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73818112->73822207] (req 73818112->73822207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.854650:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef4d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73818112->73822207] (req 73818112->73822207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.854659:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef4d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73818112->73822207] (req 73818112->73822207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.854666:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760767.854680:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef4d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73818112->73822207] (req 73818112->73822207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.854690:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73818112->73822207] (req 73818112->73822207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.854783:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef4d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73818112->73822207] (req 73818112->73822207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.854793:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef4d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73818112->73822207] (req 73818112->73822207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.854930:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503693824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5868us (6603us total) trans 4294985329 rc 0/0 00000100:00100000:3.0:1789760767.854941:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18033 00000100:00100000:3.0:1789760767.854951:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697962878839 time 697973651111 state 0x2 ref 0 00000100:00100000:3.0:1789760767.900845:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503694336 00000100:00100000:3.0:1789760767.900856:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503694336 00000100:00100000:3.0:1789760767.900911:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 697998414032 time 698014969096 state 0x0 ref 1 00000100:00100000:3.0:1789760767.900927:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698015229512 time 698014969096 state 0x1 ref 1 00000100:00100000:3.0:1789760767.900957:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698015229512 time 698025564095 state 0x0 ref 1 00000100:00100000:3.0:1789760767.900969:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503694336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760767.901016:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73834496, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23368 00010000:00010000:3.0:1789760767.901130:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.901147:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760767.901152:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.901158:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.901173:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.908876:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73834496 -> 73838592 00002000:00010000:0.0:1789760767.908881:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144232 -> 144240 00010000:00010000:0.0:1789760767.908894:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.908906:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.908914:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.908921:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.908925:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.908934:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.908954:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.908966:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef4ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73834496->73838591] (req 73834496->73838591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.909207:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503694336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8234us (8428us total) trans 4294985333 rc 0/0 00000100:00100000:0.0:1789760767.909219:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18037 00000100:00100000:0.0:1789760767.909228:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698015229512 time 698025564095 state 0x2 ref 0 00002000:00010000:0.0:1789760767.937451:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73842688 -> 73846784 00002000:00010000:0.0:1789760767.937456:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144248 -> 144256 00010000:00010000:0.0:1789760767.937470:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083ef4fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73842688->73846783] (req 73842688->73846783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.937481:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083ef4fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73842688->73846783] (req 73842688->73846783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.937490:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083ef4fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73842688->73846783] (req 73842688->73846783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.937497:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.937501:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083ef4fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73842688->73846783] (req 73842688->73846783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.937510:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083ef4fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73842688->73846783] (req 73842688->73846783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.937535:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083ef4fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73842688->73846783] (req 73842688->73846783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.937544:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083ef4fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73842688->73846783] (req 73842688->73846783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.937623:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503694592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9737us (10011us total) trans 4294985335 rc 0/0 00000100:00100000:0.0:1789760767.937633:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18039 00000100:00100000:0.0:1789760767.937643:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698039231286 time 698052477278 state 0x2 ref 0 00000100:00100000:0.0:1789760767.945611:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503694720 00000100:00100000:0.0:1789760767.945626:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503694720 00000100:00100000:0.0:1789760767.946220:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698039231286 time 698052477278 state 0x0 ref 1 00000100:00100000:0.0:1789760767.946237:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698052737694 time 698052477278 state 0x1 ref 1 00000100:00100000:0.0:1789760767.946321:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698052737694 time 698070926324 state 0x0 ref 1 00000100:00100000:0.0:1789760767.946339:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503694720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.946385:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73846784, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23380 00010000:00010000:0.0:1789760767.946434:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.946450:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760767.946456:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.946462:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.946477:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.953936:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73846784 -> 73850880 00002000:00010000:0.0:1789760767.953943:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144256 -> 144264 00010000:00010000:0.0:1789760767.953958:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.953971:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.953980:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.953988:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.953992:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.954002:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.954020:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.954032:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef504 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73846784->73850879] (req 73846784->73850879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.954125:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503694720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7788us (11319us total) trans 4294985336 rc 0/0 00000100:00100000:0.0:1789760767.954135:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18040 00000100:00100000:0.0:1789760767.954145:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698052737694 time 698070926324 state 0x2 ref 0 00000100:00100000:0.0:1789760767.958275:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503694848 00000100:00100000:0.0:1789760767.958285:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503694848 00000100:00100000:0.0:1789760767.958349:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698052737694 time 698070926324 state 0x0 ref 1 00000100:00100000:0.0:1789760767.958364:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698071186740 time 698070926324 state 0x1 ref 1 00000100:00100000:0.0:1789760767.958435:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698071186740 time 698083041001 state 0x0 ref 1 00000100:00100000:0.0:1789760767.958448:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503694848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.958496:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73850880, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23388 00010000:00010000:0.0:1789760767.958506:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.958522:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760767.958526:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.958532:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.958549:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.970199:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73850880 -> 73854976 00002000:00010000:0.0:1789760767.970205:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144264 -> 144272 00010000:00010000:0.0:1789760767.970217:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.970237:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.970245:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.970252:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760767.970257:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.970267:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.970286:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.970294:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef50b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73850880->73854975] (req 73850880->73854975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.970387:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503694848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11936us (12887us total) trans 4294985337 rc 0/0 00000100:00100000:0.0:1789760767.970449:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18041 00000100:00100000:0.0:1789760767.970459:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698071186740 time 698083041001 state 0x2 ref 0 00000100:00100000:0.0:1789760767.988567:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503695104 00000100:00100000:0.0:1789760767.988577:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503695104 00000100:00100000:0.0:1789760767.988633:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698083301417 time 698099650455 state 0x0 ref 1 00000100:00100000:0.0:1789760767.988649:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698099910871 time 698099650455 state 0x1 ref 1 00000100:00100000:0.0:1789760767.988691:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698099910871 time 698113297267 state 0x0 ref 1 00000100:00100000:0.0:1789760767.988704:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503695104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760767.988750:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73859072, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23398 00010000:00010000:0.0:1789760767.988760:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef519 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.988774:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760767.988778:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.988785:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef519 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73859072->73863167] (req 73859072->73863167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.988798:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef519 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73859072->73863167] (req 73859072->73863167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.995427:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985339 is committed 00000100:00100000:0.0:1789760768.004020:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503695232 00000100:00100000:0.0:1789760768.004029:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503695232 00000100:00100000:0.0:1789760768.004097:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698099910871 time 698113297267 state 0x0 ref 1 00000100:00100000:0.0:1789760768.004113:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698113557683 time 698113297267 state 0x1 ref 1 00000100:00100000:0.0:1789760768.004148:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698113557683 time 698128755118 state 0x0 ref 1 00000100:00100000:0.0:1789760768.004162:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503695232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.004207:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73863168, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x233a0 00010000:00010000:0.0:1789760768.004216:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.004232:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.004237:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.004243:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.004256:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.009106:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73863168 -> 73867264 00002000:00010000:0.0:1789760768.009111:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144288 -> 144296 00010000:00010000:0.0:1789760768.009124:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.009137:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.009145:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.009152:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.009156:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.009164:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.009183:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.009189:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083ef520 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73863168->73867263] (req 73863168->73867263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.009276:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503695232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5111us (5531us total) trans 4294985340 rc 0/0 00000100:00100000:0.0:1789760768.009286:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18044 00000100:00100000:0.0:1789760768.009295:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698113557683 time 698128755118 state 0x2 ref 0 00000100:00100000:0.0:1789760768.014848:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.019855:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985341 is committed 00000100:00100000:3.0:1789760768.054504:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760768.066952:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760768.075258:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503695872 00000100:00100000:0.0:1789760768.075269:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503695872 00000100:00100000:0.0:1789760768.075318:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698179633010 time 698191751300 state 0x0 ref 1 00000100:00100000:0.0:1789760768.075333:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698192011716 time 698191751300 state 0x1 ref 1 00000100:00100000:0.0:1789760768.075363:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698192011716 time 698199970540 state 0x0 ref 1 00000100:00100000:0.0:1789760768.075376:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503695872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.075452:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73883648, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x233c8 00010000:00010000:0.0:1789760768.075460:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.075476:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.075480:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.075487:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.075501:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.079563:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73883648 -> 73887744 00002000:00010000:0.0:1789760768.079571:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144328 -> 144336 00010000:00010000:0.0:1789760768.079588:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.079611:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.079621:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.079630:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.079636:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.079646:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.079781:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.079792:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef543 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73883648->73887743] (req 73883648->73887743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.080259:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503695872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4877us (5095us total) trans 4294985345 rc 0/0 00000100:00100000:0.0:1789760768.080273:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18049 00000100:00100000:0.0:1789760768.080286:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698192011716 time 698199970540 state 0x2 ref 0 00000100:00100000:3.0:1789760768.084655:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760768.104138:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503696128 00000100:00100000:0.0:1789760768.104152:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503696128 00000100:00100000:0.0:1789760768.104342:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698200230956 time 698209575546 state 0x0 ref 1 00000100:00100000:0.0:1789760768.104364:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698209835962 time 698209575546 state 0x1 ref 1 00000100:00100000:0.0:1789760768.104625:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698209835962 time 698229229024 state 0x0 ref 1 00000100:00100000:0.0:1789760768.104643:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503696128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.104897:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73891840, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x233d8 00010000:00010000:0.0:1789760768.104910:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.104931:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.104936:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.104943:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.104961:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.114562:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73891840 -> 73895936 00002000:00010000:0.0:1789760768.114577:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144344 -> 144352 00010000:00010000:0.0:1789760768.114596:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114612:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114627:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114635:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.114640:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114650:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114688:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114698:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083ef551 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73891840->73895935] (req 73891840->73895935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.114837:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503696128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10192us (10832us total) trans 4294985347 rc 0/0 00000100:00100000:0.0:1789760768.114849:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18051 00000100:00100000:0.0:1789760768.114861:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698209835962 time 698229229024 state 0x2 ref 0 00000100:00100000:3.0:1789760768.135602:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.141136:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985349 is committed 00002000:00010000:3.0:1789760768.141549:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73900032 -> 73904128 00002000:00010000:3.0:1789760768.141555:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144360 -> 144368 00010000:00010000:3.0:1789760768.141572:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ef55f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73900032->73904127] (req 73900032->73904127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.141589:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ef55f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73900032->73904127] (req 73900032->73904127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.141600:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ef55f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73900032->73904127] (req 73900032->73904127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.141609:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.141614:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ef55f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73900032->73904127] (req 73900032->73904127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.141625:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ef55f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73900032->73904127] (req 73900032->73904127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.141651:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ef55f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73900032->73904127] (req 73900032->73904127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.141659:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083ef55f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73900032->73904127] (req 73900032->73904127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.141851:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503696384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5839us (6255us total) trans 4294985349 rc 0/0 00000100:00100000:3.0:1789760768.141866:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18053 00000100:00100000:3.0:1789760768.141879:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698245012358 time 698260599252 state 0x2 ref 0 00000100:00100000:0.0:1789760768.149230:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.156182:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985350 is committed 00000100:00100000:0.0:1789760768.162768:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503696640 00000100:00100000:0.0:1789760768.162783:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503696640 00000100:00100000:0.0:1789760768.162864:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698260859668 time 698275145042 state 0x0 ref 1 00000100:00100000:0.0:1789760768.162886:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698275405458 time 698275145042 state 0x1 ref 1 00000100:00100000:0.0:1789760768.162966:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698275405458 time 698287571424 state 0x0 ref 1 00000100:00100000:0.0:1789760768.162981:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503696640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.163046:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73908224, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x233f8 00010000:00010000:0.0:1789760768.163058:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.163091:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.163096:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.163104:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.163121:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.170659:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985351 is committed 00002000:00010000:0.0:1789760768.170859:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73908224 -> 73912320 00002000:00010000:0.0:1789760768.170866:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144376 -> 144384 00010000:00010000:0.0:1789760768.170884:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.170900:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.170911:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.170921:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.170927:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.170937:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.170963:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.170972:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef56d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73908224->73912319] (req 73908224->73912319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.171146:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503696640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8161us (8778us total) trans 4294985351 rc 0/0 00000100:00100000:0.0:1789760768.171160:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18055 00000100:00100000:0.0:1789760768.171174:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698275405458 time 698287571424 state 0x2 ref 0 00000100:00100000:3.0:1789760768.188527:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503696896 00000100:00100000:3.0:1789760768.188542:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503696896 00000100:00100000:3.0:1789760768.188877:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698287831840 time 698301174805 state 0x0 ref 1 00000100:00100000:3.0:1789760768.188894:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698301435221 time 698301174805 state 0x1 ref 1 00000100:00100000:3.0:1789760768.188929:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698301435221 time 698313535519 state 0x0 ref 1 00000100:00100000:3.0:1789760768.188943:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503696896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.189001:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73916416, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23408 00010000:00010000:3.0:1789760768.189011:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.189204:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.189209:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.189216:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.189231:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.195386:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73916416 -> 73920512 00002000:00010000:3.0:1789760768.195509:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144392 -> 144400 00010000:00010000:3.0:1789760768.195524:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.195537:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.195546:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.195554:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.195560:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.196192:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.196214:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.196222:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef57b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73916416->73920511] (req 73916416->73920511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.203227:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503697024 00000100:00100000:0.0:1789760768.203237:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503697024 00000100:00100000:0.0:1789760768.203292:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698301435221 time 698313535519 state 0x0 ref 1 00000100:00100000:0.0:1789760768.203306:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698313795935 time 698313535519 state 0x1 ref 1 00000100:00100000:0.0:1789760768.203484:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698313795935 time 698328088900 state 0x0 ref 1 00000100:00100000:0.0:1789760768.203501:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503697024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.203546:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73920512, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23410 00010000:00010000:0.0:1789760768.203563:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.203579:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.203584:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.203591:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.203604:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.209865:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73920512 -> 73924608 00002000:00010000:0.0:1789760768.209870:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144400 -> 144408 00010000:00010000:0.0:1789760768.209883:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.210014:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.210219:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.210227:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.210233:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.210242:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.210263:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.210270:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef582 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73920512->73924607] (req 73920512->73924607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.210366:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503697024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6863us (7492us total) trans 4294985354 rc 0/0 00000100:00100000:0.0:1789760768.210377:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18058 00000100:00100000:0.0:1789760768.210386:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698313795935 time 698328088900 state 0x2 ref 0 00000100:00100000:0.0:1789760768.234145:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503697280 00000100:00100000:0.0:1789760768.234155:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503697280 00000100:00100000:0.0:1789760768.234209:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698328349316 time 698341378942 state 0x0 ref 1 00000100:00100000:0.0:1789760768.234224:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698341639358 time 698341378942 state 0x1 ref 1 00000100:00100000:0.0:1789760768.234255:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698341639358 time 698358862326 state 0x0 ref 1 00000100:00100000:0.0:1789760768.234269:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503697280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.234314:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73928704, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23420 00010000:00010000:0.0:1789760768.234324:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.234341:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.234346:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.234352:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.234365:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.239773:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73928704 -> 73932800 00002000:00010000:0.0:1789760768.239778:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144416 -> 144424 00010000:00010000:0.0:1789760768.239792:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.239805:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.239815:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.239823:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.239827:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.239836:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.239857:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.239865:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef590 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73928704->73932799] (req 73928704->73932799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.239985:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503697280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5715us (6020us total) trans 4294985356 rc 0/0 00000100:00100000:0.0:1789760768.239995:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18060 00000100:00100000:0.0:1789760768.240003:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698341639358 time 698358862326 state 0x2 ref 0 00000100:00100000:3.0:1789760768.243269:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503697408 00000100:00100000:3.0:1789760768.243282:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503697408 00000100:00100000:3.0:1789760768.243336:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698341639358 time 698358862326 state 0x0 ref 1 00000100:00100000:3.0:1789760768.243351:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698359122742 time 698358862326 state 0x1 ref 1 00000100:00100000:3.0:1789760768.243464:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698359122742 time 698368068950 state 0x0 ref 1 00000100:00100000:3.0:1789760768.243476:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503697408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.243523:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73932800, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23428 00010000:00010000:3.0:1789760768.243532:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.243545:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.243549:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.243555:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.243569:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.249665:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985357 is committed 00002000:00010000:3.0:1789760768.251601:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73932800 -> 73936896 00002000:00010000:3.0:1789760768.251612:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144424 -> 144432 00010000:00010000:3.0:1789760768.251625:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.251637:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.251653:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.251661:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.251665:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.251686:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.251705:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.252180:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef597 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73932800->73936895] (req 73932800->73936895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.252303:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503697408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8823us (9412us total) trans 4294985357 rc 0/0 00000100:00100000:3.0:1789760768.252314:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18061 00000100:00100000:3.0:1789760768.252324:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698359122742 time 698368068950 state 0x2 ref 0 00000001:00080000:0.0:1789760768.266450:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985358 is committed 00000100:00100000:0.0:1789760768.287351:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503697664 00000100:00100000:0.0:1789760768.287361:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503697664 00000100:00100000:0.0:1789760768.287451:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698368329366 time 698385963980 state 0x0 ref 1 00000100:00100000:0.0:1789760768.287465:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698386224396 time 698385963980 state 0x1 ref 1 00000100:00100000:0.0:1789760768.287498:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698386224396 time 698412104380 state 0x0 ref 1 00000100:00100000:0.0:1789760768.287515:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503697664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.287567:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73940992, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23438 00010000:00010000:0.0:1789760768.287577:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.287597:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.287601:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.287607:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73940992->73945087] (req 73940992->73945087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.287619:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73940992->73945087] (req 73940992->73945087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.296368:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985359 is committed 00000100:00100000:0.0:1789760768.315112:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503697792 00000100:00100000:0.0:1789760768.315123:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503697792 00000100:00100000:0.0:1789760768.315180:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698386224396 time 698412104380 state 0x0 ref 1 00000100:00100000:0.0:1789760768.315205:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698412364796 time 698412104380 state 0x1 ref 1 00000100:00100000:0.0:1789760768.315236:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698412364796 time 698439842630 state 0x0 ref 1 00000100:00100000:0.0:1789760768.315247:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503697792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.315295:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73945088, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23440 00010000:00010000:0.0:1789760768.315305:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.315321:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.315326:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.315332:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.315345:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.322711:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985360 is committed 00002000:00010000:0.0:1789760768.322843:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73945088 -> 73949184 00002000:00010000:0.0:1789760768.322849:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144448 -> 144456 00010000:00010000:0.0:1789760768.322861:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.322875:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.322885:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.322893:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.322898:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.322908:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.322928:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.322936:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73945088->73949183] (req 73945088->73949183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.323028:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503697792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7778us (8014us total) trans 4294985360 rc 0/0 00000100:00100000:0.0:1789760768.323043:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18064 00000100:00100000:0.0:1789760768.323053:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698412364796 time 698439842630 state 0x2 ref 0 00000100:00100000:3.0:1789760768.333193:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503697920 00000100:00100000:3.0:1789760768.333204:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503697920 00000100:00100000:3.0:1789760768.333256:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698412364796 time 698439842630 state 0x0 ref 1 00000100:00100000:3.0:1789760768.333274:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698440103046 time 698439842630 state 0x1 ref 1 00000100:00100000:3.0:1789760768.333305:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698440103046 time 698457911663 state 0x0 ref 1 00000100:00100000:3.0:1789760768.333318:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503697920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.333364:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73949184, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23448 00010000:00010000:3.0:1789760768.333374:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333432:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.333437:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.333442:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.333455:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.341280:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985361 is committed 00002000:00010000:3.0:1789760768.341786:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73949184 -> 73953280 00002000:00010000:3.0:1789760768.341792:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144456 -> 144464 00010000:00010000:3.0:1789760768.341805:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.341818:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.341827:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.341835:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.341839:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.341849:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.341870:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.341877:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73949184->73953279] (req 73949184->73953279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.341980:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503697920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8661us (8880us total) trans 4294985361 rc 0/0 00000100:00100000:3.0:1789760768.341990:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18065 00000100:00100000:3.0:1789760768.341999:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698440103046 time 698457911663 state 0x2 ref 0 00000100:00100000:0.0:1789760768.345194:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503698048 00000100:00100000:0.0:1789760768.345205:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503698048 00000100:00100000:0.0:1789760768.345282:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698440103046 time 698457911663 state 0x0 ref 1 00000100:00100000:0.0:1789760768.345297:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698458172079 time 698457911663 state 0x1 ref 1 00000100:00100000:0.0:1789760768.345327:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698458172079 time 698469933990 state 0x0 ref 1 00000100:00100000:0.0:1789760768.345339:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503698048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.345384:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73953280, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23450 00010000:00010000:0.0:1789760768.345436:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.345452:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.345457:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.345462:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73953280->73957375] (req 73953280->73957375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.345475:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef5ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73953280->73957375] (req 73953280->73957375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.352634:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985362 is committed 00000001:00080000:0.0:1789760768.364150:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985363 is committed 00000100:00100000:3.0:1789760768.378805:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.386887:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985364 is committed 00000100:00100000:0.0:1789760768.399870:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760768.399954:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503698432 00000100:00100000:3.0:1789760768.399964:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503698432 00000100:00100000:3.0:1789760768.400015:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698483935596 time 698503869745 state 0x0 ref 1 00000100:00100000:3.0:1789760768.400031:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698504130161 time 698503869745 state 0x1 ref 1 00000100:00100000:3.0:1789760768.400101:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698504130161 time 698524706652 state 0x0 ref 1 00000100:00100000:3.0:1789760768.400114:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503698432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.400163:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73965568, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23468 00010000:00010000:3.0:1789760768.400172:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.400188:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.400192:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.400198:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.400211:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.404148:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985365 is committed 00002000:00010000:3.0:1789760768.404325:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73965568 -> 73969664 00002000:00010000:3.0:1789760768.404331:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144488 -> 144496 00010000:00010000:3.0:1789760768.404344:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.404356:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.404365:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.404373:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.404378:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.404498:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.404520:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.404527:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73965568->73969663] (req 73965568->73969663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.404904:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503698432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4786us (5037us total) trans 4294985365 rc 0/0 00000100:00100000:3.0:1789760768.404915:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18069 00000100:00100000:3.0:1789760768.404925:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698504130161 time 698524706652 state 0x2 ref 0 00000001:00080000:0.0:1789760768.419713:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985366 is committed 00000001:00080000:0.0:1789760768.438336:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985367 is committed 00002000:00010000:3.0:1789760768.438791:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73973760 -> 73977856 00002000:00010000:3.0:1789760768.438797:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144504 -> 144512 00010000:00010000:3.0:1789760768.438960:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef5dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73973760->73977855] (req 73973760->73977855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.438974:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef5dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73973760->73977855] (req 73973760->73977855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.438984:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef5dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73973760->73977855] (req 73973760->73977855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.438997:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.439002:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef5dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73973760->73977855] (req 73973760->73977855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.439012:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef5dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73973760->73977855] (req 73973760->73977855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.439032:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef5dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73973760->73977855] (req 73973760->73977855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.439040:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083ef5dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73973760->73977855] (req 73973760->73977855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.439145:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503698688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6589us (7002us total) trans 4294985367 rc 0/0 00000100:00100000:3.0:1789760768.439155:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18071 00000100:00100000:3.0:1789760768.439165:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698535678365 time 698557139212 state 0x2 ref 0 00000100:00100000:0.0:1789760768.443119:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.447306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985368 is committed 00000100:00100000:3.0:1789760768.452072:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503698944 00000100:00100000:3.0:1789760768.452082:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503698944 00000100:00100000:3.0:1789760768.452127:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698557399628 time 698567933226 state 0x0 ref 1 00000100:00100000:3.0:1789760768.452141:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698568193642 time 698567933226 state 0x1 ref 1 00000100:00100000:3.0:1789760768.452203:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698568193642 time 698576809967 state 0x0 ref 1 00000100:00100000:3.0:1789760768.452218:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503698944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.452260:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73981952, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23488 00010000:00010000:3.0:1789760768.452280:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.452296:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.452300:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.452305:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.452318:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.456839:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985369 is committed 00002000:00010000:3.0:1789760768.456953:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73981952 -> 73986048 00002000:00010000:3.0:1789760768.456958:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144520 -> 144528 00010000:00010000:3.0:1789760768.456970:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.456983:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.456992:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.456999:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.457004:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.457013:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.457032:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.457039:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73981952->73986047] (req 73981952->73986047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.457177:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503698944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4955us (5188us total) trans 4294985369 rc 0/0 00000100:00100000:3.0:1789760768.457189:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18073 00000100:00100000:3.0:1789760768.457198:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698568193642 time 698576809967 state 0x2 ref 0 00000001:00080000:0.0:1789760768.466306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985370 is committed 00000100:00100000:0.0:1789760768.472132:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760768.472212:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503699200 00000100:00100000:3.0:1789760768.472221:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503699200 00000100:00100000:3.0:1789760768.472272:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698577070383 time 698585765966 state 0x0 ref 1 00000100:00100000:3.0:1789760768.472287:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698586026382 time 698585765966 state 0x1 ref 1 00000100:00100000:3.0:1789760768.472313:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698586026382 time 698596919681 state 0x0 ref 1 00000100:00100000:3.0:1789760768.472325:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503699200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.472372:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73990144, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23498 00010000:00010000:3.0:1789760768.472381:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.472441:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.472446:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.472452:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.472466:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.479216:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985371 is committed 00002000:00010000:3.0:1789760768.479613:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73990144 -> 73994240 00002000:00010000:3.0:1789760768.479618:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144536 -> 144544 00010000:00010000:3.0:1789760768.479630:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.479642:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.479650:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.479657:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.479662:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.479684:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.479703:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.479856:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef5f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73990144->73994239] (req 73990144->73994239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.479961:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503699200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7632us (7832us total) trans 4294985371 rc 0/0 00000100:00100000:3.0:1789760768.479971:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18075 00000100:00100000:3.0:1789760768.479981:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698586026382 time 698596919681 state 0x2 ref 0 00000100:00100000:0.0:1789760768.483794:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503699328 00000100:00100000:0.0:1789760768.483805:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503699328 00000100:00100000:0.0:1789760768.483861:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698586026382 time 698596919681 state 0x0 ref 1 00000100:00100000:0.0:1789760768.483876:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698597180097 time 698596919681 state 0x1 ref 1 00000100:00100000:0.0:1789760768.483906:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698597180097 time 698608513148 state 0x0 ref 1 00000100:00100000:0.0:1789760768.483917:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503699328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.483961:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73994240, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x234a0 00010000:00010000:0.0:1789760768.483969:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef600 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.483987:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.483991:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.483997:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73994240->73998335] (req 73994240->73998335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.484009:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73994240->73998335] (req 73994240->73998335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.492614:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985372 is committed 00000100:00100000:3.0:1789760768.497440:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503699456 00000100:00100000:3.0:1789760768.497452:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503699456 00000100:00100000:3.0:1789760768.497511:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698597180097 time 698608513148 state 0x0 ref 1 00000100:00100000:3.0:1789760768.497528:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698608773564 time 698608513148 state 0x1 ref 1 00000100:00100000:3.0:1789760768.497549:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698608773564 time 698622155914 state 0x0 ref 1 00000100:00100000:3.0:1789760768.497561:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503699456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.497608:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 73998336, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x234a8 00010000:00010000:3.0:1789760768.497618:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.497633:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.497638:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.497645:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.497659:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.503568:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985373 is committed 00002000:00010000:3.0:1789760768.503690:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 73998336 -> 74002432 00002000:00010000:3.0:1789760768.503696:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144552 -> 144560 00010000:00010000:3.0:1789760768.503714:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.503728:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.503737:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.503745:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.503749:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.503760:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.503778:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.503791:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef607 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [73998336->74002431] (req 73998336->74002431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.503885:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503699456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6321us (6566us total) trans 4294985373 rc 0/0 00000100:00100000:3.0:1789760768.503894:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18077 00000100:00100000:3.0:1789760768.503903:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698608773564 time 698622155914 state 0x2 ref 0 00000100:00100000:0.0:1789760768.508904:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.517333:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985374 is committed 00000100:00100000:0.0:1789760768.524987:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503699712 00000100:00100000:0.0:1789760768.524998:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503699712 00000100:00100000:0.0:1789760768.525051:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698622416330 time 698633719074 state 0x0 ref 1 00000100:00100000:0.0:1789760768.525065:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698633979490 time 698633719074 state 0x1 ref 1 00000100:00100000:0.0:1789760768.525095:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698633979490 time 698649702204 state 0x0 ref 1 00000100:00100000:0.0:1789760768.525107:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503699712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.525152:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74006528, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x234b8 00010000:00010000:0.0:1789760768.525161:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083ef615 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.525176:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760768.525181:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.525188:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083ef615 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74006528->74010623] (req 74006528->74010623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.525203:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083ef615 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74006528->74010623] (req 74006528->74010623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.530846:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760768.531021:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985375 is committed 00000001:00080000:3.0:1789760768.550075:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985376 is committed 00000100:00100000:0.0:1789760768.556861:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760768.557028:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503699968 00000100:00100000:3.0:1789760768.557040:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503699968 00000100:00100000:3.0:1789760768.557093:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698649962620 time 698666961304 state 0x0 ref 1 00000100:00100000:3.0:1789760768.557109:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698667221720 time 698666961304 state 0x1 ref 1 00000100:00100000:3.0:1789760768.557142:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698667221720 time 698681748298 state 0x0 ref 1 00000100:00100000:3.0:1789760768.557153:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503699968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.557200:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74014720, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x234c8 00010000:00010000:3.0:1789760768.557210:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.557233:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.557237:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.557244:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.557258:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.562636:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985377 is committed 00002000:00010000:3.0:1789760768.562974:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74014720 -> 74018816 00002000:00010000:3.0:1789760768.562978:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144584 -> 144592 00010000:00010000:3.0:1789760768.562989:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.562998:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.563004:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.563009:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.563096:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.563103:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.563121:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.563126:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef623 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74014720->74018815] (req 74014720->74018815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.563267:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503699968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6108us (6408us total) trans 4294985377 rc 0/0 00000100:00100000:3.0:1789760768.563277:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18081 00000100:00100000:3.0:1789760768.563287:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698667221720 time 698681748298 state 0x2 ref 0 00000100:00100000:3.0:1789760768.567362:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.572449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985378 is committed 00000100:00100000:3.0:1789760768.576562:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503700224 00000100:00100000:3.0:1789760768.576580:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503700224 00000100:00100000:3.0:1789760768.576633:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698682008714 time 698692584856 state 0x0 ref 1 00000100:00100000:3.0:1789760768.576649:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698692845272 time 698692584856 state 0x1 ref 1 00000100:00100000:3.0:1789760768.576688:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698692845272 time 698701295048 state 0x0 ref 1 00000100:00100000:3.0:1789760768.576700:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503700224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.576756:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74022912, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x234d8 00010000:00010000:3.0:1789760768.576765:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.576785:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.576790:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.576796:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.576809:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.583431:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985379 is committed 00002000:00010000:3.0:1789760768.583632:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74022912 -> 74027008 00002000:00010000:3.0:1789760768.583638:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144600 -> 144608 00010000:00010000:3.0:1789760768.583656:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.583791:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.583798:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.583803:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.583807:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.583813:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.583882:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.583888:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef631 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74022912->74027007] (req 74022912->74027007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.583972:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503700224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7269us (7690us total) trans 4294985379 rc 0/0 00000100:00100000:3.0:1789760768.583982:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18083 00000100:00100000:3.0:1789760768.583992:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698692845272 time 698701295048 state 0x2 ref 0 00000100:00100000:0.0:1789760768.586773:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760768.586908:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503700352 00000100:00100000:3.0:1789760768.586917:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503700352 00000100:00100000:3.0:1789760768.586970:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698692845272 time 698701295048 state 0x0 ref 1 00000100:00100000:3.0:1789760768.586985:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698701555464 time 698701295048 state 0x1 ref 1 00000100:00100000:3.0:1789760768.587018:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698701555464 time 698711624543 state 0x0 ref 1 00000100:00100000:3.0:1789760768.587030:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503700352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.587076:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74027008, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x234e0 00010000:00010000:3.0:1789760768.587086:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.587101:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.587105:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.587111:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.587126:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.596449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985380 is committed 00002000:00010000:3.0:1789760768.596678:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74027008 -> 74031104 00002000:00010000:3.0:1789760768.596683:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144608 -> 144616 00010000:00010000:3.0:1789760768.596697:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.596709:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.596718:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.596725:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.596730:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.596738:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.596757:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.596833:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef638 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74027008->74031103] (req 74027008->74031103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.596943:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503700352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9910us (10173us total) trans 4294985380 rc 0/0 00000100:00100000:3.0:1789760768.596954:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18084 00000100:00100000:3.0:1789760768.596974:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698701555464 time 698711624543 state 0x2 ref 0 00000100:00100000:3.0:1789760768.602527:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503700480 00000100:00100000:3.0:1789760768.602537:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503700480 00000100:00100000:3.0:1789760768.602660:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698701555464 time 698711624543 state 0x0 ref 1 00000100:00100000:3.0:1789760768.602907:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698711884959 time 698711624543 state 0x1 ref 1 00000100:00100000:3.0:1789760768.603053:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698711884959 time 698727658107 state 0x0 ref 1 00000100:00100000:3.0:1789760768.603067:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503700480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.603117:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74031104, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x234e8 00010000:00010000:3.0:1789760768.603386:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.603432:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.603436:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.603442:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.603461:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.611890:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985381 is committed 00002000:00010000:3.0:1789760768.612223:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74031104 -> 74035200 00002000:00010000:3.0:1789760768.612229:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144616 -> 144624 00010000:00010000:3.0:1789760768.612241:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.612254:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.612263:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.612336:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.612341:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.612350:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.612369:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.612375:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef63f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74031104->74035199] (req 74031104->74035199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.627381:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985382 is committed 00000100:00100000:0.0:1789760768.634834:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.640635:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985383 is committed 00000100:00100000:3.0:1789760768.646436:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503700864 00000100:00100000:3.0:1789760768.646447:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503700864 00000100:00100000:3.0:1789760768.646502:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698745698219 time 698759655373 state 0x0 ref 1 00000100:00100000:3.0:1789760768.646516:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698759915789 time 698759655373 state 0x1 ref 1 00000100:00100000:3.0:1789760768.646547:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698759915789 time 698771153251 state 0x0 ref 1 00000100:00100000:3.0:1789760768.646560:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503700864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.646811:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74043392, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23500 00010000:00010000:3.0:1789760768.646820:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646846:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.646851:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.646858:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.646873:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.654756:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985384 is committed 00002000:00010000:3.0:1789760768.654898:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74043392 -> 74047488 00002000:00010000:3.0:1789760768.654903:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144640 -> 144648 00010000:00010000:3.0:1789760768.654914:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.654932:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.654942:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.654948:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.654952:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.654960:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.654978:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.654985:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef654 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74043392->74047487] (req 74043392->74047487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.655083:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503700864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8520us (8770us total) trans 4294985384 rc 0/0 00000100:00100000:3.0:1789760768.655092:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18088 00000100:00100000:3.0:1789760768.655100:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698759915789 time 698771153251 state 0x2 ref 0 00000100:00100000:3.0:1789760768.658488:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.665383:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985385 is committed 00000100:00100000:0.0:1789760768.670044:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.675948:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985386 is committed 00000100:00100000:3.0:1789760768.682514:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503701248 00000100:00100000:3.0:1789760768.682524:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503701248 00000100:00100000:3.0:1789760768.682580:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698783674369 time 698794919091 state 0x0 ref 1 00000100:00100000:3.0:1789760768.682595:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698795179507 time 698794919091 state 0x1 ref 1 00000100:00100000:3.0:1789760768.682680:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698795179507 time 698807285151 state 0x0 ref 1 00000100:00100000:3.0:1789760768.682694:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503701248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.682752:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74055680, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x23518 00010000:00010000:3.0:1789760768.682761:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef669 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.682777:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760768.682781:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.682788:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74055680->74059775] (req 74055680->74059775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.682801:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74055680->74059775] (req 74055680->74059775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.687527:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985387 is committed 00000100:00100000:3.0:1789760768.691627:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.698502:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985388 is committed 00000100:00100000:3.0:1789760768.702591:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.716843:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985389 is committed 00000100:00100000:3.0:1789760768.722609:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523604096:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.722694:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.722761:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e19180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523604608:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.722794:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.722832:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e1b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523605120:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.722860:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.722886:0:3636:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:3.0:1789760768.722894:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:afc717a1-11ba-4ef0-ab10-103aa692ca98:3636:1876699523603712:0@lo:400:kworker.0 00000100:00100000:0.0:1789760768.724723:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523604096 00000100:00100000:0.0:1789760768.724737:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523604096 00000100:00100000:0.0:1789760768.724799:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1789760768.724811:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3636:x1876699523604096:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.727355:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.727492:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e19880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3636:1876699523604224:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.727548:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.727571:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e18a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523604736:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.727596:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.727617:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e1b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523605248:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.727637:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.727690:0:6505:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523603712 00000100:00100000:3.0:1789760768.727698:0:6505:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523603712 00000100:00100000:3.0:1789760768.727750:0:6505:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:3.0:1789760768.727758:0:6505:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3636:x1876699523603712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.727809:0:6505:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3636:x1876699523603712:12345-0@lo:400:kworker.0 Request processed in 52us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.727816:0:6505:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:3.0:1789760768.727849:0:8429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523604736 00000100:00100000:3.0:1789760768.727854:0:8429:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523604736 00000100:00100000:3.0:1789760768.727888:0:8429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523605248 00000100:00100000:3.0:1789760768.727891:0:8429:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523605248 00000100:00100000:3.0:1789760768.727912:0:8429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1789760768.727919:0:8429:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523604736:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.727955:0:8429:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523604736:12345-0@lo:400:kworker.0 Request processed in 35us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.727962:0:8429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:0.0:1789760768.728686:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3636:x1876699523604096:12345-0@lo:400:kworker.0 Request processed in 3877us (5991us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760768.728694:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1789760768.729048:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523604352 00000100:00100000:0.0:1789760768.729053:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523604352 00000100:00100000:0.0:1789760768.729083:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760768.729090:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0777100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523604352:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760768.729131:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0777100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523604352:12345-0@lo:400:kworker.0 Request processed in 42us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760768.729137:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760768.729290:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:0.0:1789760768.729296:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523603840:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760768.729381:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1789760768.729432:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523603840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 136us (774us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760768.729439:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:0.0:1789760768.729451:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:0.0:1789760768.729456:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523605504:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760768.729465:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1789760768.729483:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523605504:12345-0@lo:13:osp-pre-1-1.0 Request processed in 27us (480us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760768.729488:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:3.0:1789760768.731934:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:afc717a1-11ba-4ef0-ab10-103aa692ca98:3636:1876699523603712:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.731947:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e19880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3636:1876699523604224:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.731957:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e18a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523604736:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.731967:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e1b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523605248:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.731976:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e19180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523604608:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.731986:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e1b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523605120:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.731996:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523604096:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.732067:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.741100:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985390 is committed 00000100:00100000:3.0:1789760768.748887:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.755695:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985391 is committed 00000100:00100000:0.0:1789760768.764481:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.777995:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985392 is committed 00000100:00100000:3.0:1789760768.790224:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760768.802860:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985393 is committed 00000001:00080000:3.0:1789760768.818621:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985394 is committed 00000100:00100000:0.0:1789760768.829825:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760768.840608:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760768.852848:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74092544 -> 74096640 00002000:00010000:3.0:1789760768.852855:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144736 -> 144744 00010000:00010000:3.0:1789760768.852869:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ef6a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74092544->74096639] (req 74092544->74096639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.852887:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ef6a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74092544->74096639] (req 74092544->74096639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.852895:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ef6a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74092544->74096639] (req 74092544->74096639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.852902:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.852907:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ef6a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74092544->74096639] (req 74092544->74096639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.852916:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ef6a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74092544->74096639] (req 74092544->74096639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.852935:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ef6a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74092544->74096639] (req 74092544->74096639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.852942:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083ef6a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74092544->74096639] (req 74092544->74096639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.853038:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503702400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12209us (12433us total) trans 4294985396 rc 0/0 00000100:00100000:3.0:1789760768.853048:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18100 00000100:00100000:3.0:1789760768.853058:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 698954886607 time 698965420909 state 0x2 ref 0 00000100:00100000:3.0:1789760768.858381:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.869977:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985397 is committed 00000100:00100000:0.0:1789760768.878992:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.889140:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985398 is committed 00000100:00100000:0.0:1789760768.892781:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760768.903604:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760768.916370:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74108928 -> 74113024 00002000:00010000:3.0:1789760768.916380:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144768 -> 144776 00010000:00010000:3.0:1789760768.916408:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef6c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74108928->74113023] (req 74108928->74113023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.916422:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef6c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74108928->74113023] (req 74108928->74113023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.916430:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef6c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74108928->74113023] (req 74108928->74113023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.916437:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.916442:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef6c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74108928->74113023] (req 74108928->74113023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.916450:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef6c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74108928->74113023] (req 74108928->74113023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.916468:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef6c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74108928->74113023] (req 74108928->74113023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.916475:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef6c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74108928->74113023] (req 74108928->74113023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.916639:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503702912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12489us (13037us total) trans 4294985400 rc 0/0 00000100:00100000:3.0:1789760768.916658:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18104 00000100:00100000:3.0:1789760768.917446:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699018200042 time 699028741823 state 0x2 ref 0 00000100:00100000:0.0:1789760768.924501:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760768.929773:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699018200042 time 699028741823 state 0x0 ref 1 00000100:00100000:0.0:1789760768.929791:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699029002239 time 699028741823 state 0x1 ref 1 00000100:00100000:0.0:1789760768.929883:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699029002239 time 699054488411 state 0x0 ref 1 00000100:00100000:0.0:1789760768.929898:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503703040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760768.929945:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74113024, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23588 00010000:00010000:0.0:1789760768.929954:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.929970:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760768.929974:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.929980:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.929992:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.935326:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74113024 -> 74117120 00002000:00010000:0.0:1789760768.935332:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144776 -> 144784 00010000:00010000:0.0:1789760768.935345:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.935357:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.935371:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.935378:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760768.935382:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.935410:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.935430:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.935438:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef6cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74113024->74117119] (req 74113024->74117119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.935534:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503703040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5634us (11039us total) trans 4294985401 rc 0/0 00000100:00100000:0.0:1789760768.935544:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18105 00000100:00100000:0.0:1789760768.935555:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699029002239 time 699054488411 state 0x2 ref 0 00000001:00080000:0.0:1789760768.944574:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985402 is committed 00000100:00100000:3.0:1789760768.951848:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.961244:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985403 is committed 00000100:00100000:0.0:1789760768.966973:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760768.967084:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503703424 00000100:00100000:3.0:1789760768.967094:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503703424 00000100:00100000:3.0:1789760768.967217:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699064697337 time 699076636693 state 0x0 ref 1 00000100:00100000:3.0:1789760768.967232:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699076897109 time 699076636693 state 0x1 ref 1 00000100:00100000:3.0:1789760768.967655:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699076897109 time 699091869934 state 0x0 ref 1 00000100:00100000:3.0:1789760768.968187:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503703424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760768.968238:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74125312, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x235a0 00010000:00010000:3.0:1789760768.968246:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.968261:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760768.968276:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.968282:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.968297:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.977232:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985404 is committed 00002000:00010000:3.0:1789760768.978189:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74125312 -> 74129408 00002000:00010000:3.0:1789760768.978196:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144800 -> 144808 00010000:00010000:3.0:1789760768.978208:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978220:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978228:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978235:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760768.978240:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978255:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978333:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978341:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74125312->74129407] (req 74125312->74129407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.978592:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503703424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10408us (11622us total) trans 4294985404 rc 0/0 00000100:00100000:3.0:1789760768.978603:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18108 00000100:00100000:3.0:1789760768.978613:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699076897109 time 699091869934 state 0x2 ref 0 00000100:00100000:0.0:1789760768.986698:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760768.997510:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985405 is committed 00000001:00080000:0.0:1789760769.008629:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985406 is committed 00000100:00100000:0.0:1789760769.014210:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.021094:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985407 is committed 00000100:00100000:0.0:1789760769.027926:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.028306:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503703936 00000100:00100000:3.0:1789760769.028316:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503703936 00000100:00100000:3.0:1789760769.028373:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699127418469 time 699139140557 state 0x0 ref 1 00000100:00100000:3.0:1789760769.028813:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699139400973 time 699139140557 state 0x1 ref 1 00000100:00100000:3.0:1789760769.028849:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699139400973 time 699153455389 state 0x0 ref 1 00000100:00100000:3.0:1789760769.028865:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503703936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.028914:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74141696, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x235c0 00010000:00010000:3.0:1789760769.028923:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.028939:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.028943:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.028950:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.028964:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.037922:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985408 is committed 00002000:00010000:3.0:1789760769.038045:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74141696 -> 74145792 00002000:00010000:3.0:1789760769.038050:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144832 -> 144840 00010000:00010000:3.0:1789760769.038064:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.038084:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.038093:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.038101:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.038105:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.038115:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.038135:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.038143:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef6fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74141696->74145791] (req 74141696->74145791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.038282:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503703936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9416us (10360us total) trans 4294985408 rc 0/0 00000100:00100000:3.0:1789760769.038292:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18112 00000100:00100000:3.0:1789760769.038302:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699139400973 time 699153455389 state 0x2 ref 0 00000100:00100000:3.0:1789760769.049660:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.057114:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985409 is committed 00000100:00100000:0.0:1789760769.064803:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.071329:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985410 is committed 00000100:00100000:0.0:1789760769.074754:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.083281:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985411 is committed 00000100:00100000:3.0:1789760769.089207:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503704448 00000100:00100000:3.0:1789760769.089219:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503704448 00000100:00100000:3.0:1789760769.089274:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699189977701 time 699199713024 state 0x0 ref 1 00000100:00100000:3.0:1789760769.089290:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699199973440 time 699199713024 state 0x1 ref 1 00000100:00100000:3.0:1789760769.089320:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699199973440 time 699213926260 state 0x0 ref 1 00000100:00100000:3.0:1789760769.089333:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503704448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.089419:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74158080, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x235e0 00010000:00010000:3.0:1789760769.089429:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.089444:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.089449:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.089456:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.089469:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.096278:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985412 is committed 00002000:00010000:3.0:1789760769.096455:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74158080 -> 74162176 00002000:00010000:3.0:1789760769.096460:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144864 -> 144872 00010000:00010000:3.0:1789760769.096471:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.096483:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.096493:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.096500:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.096505:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.096513:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.096530:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.096537:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef718 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74158080->74162175] (req 74158080->74162175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.096686:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503704448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7351us (7784us total) trans 4294985412 rc 0/0 00000100:00100000:3.0:1789760769.096698:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18116 00000100:00100000:3.0:1789760769.096721:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699199973440 time 699213926260 state 0x2 ref 0 00000100:00100000:0.0:1789760769.107077:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:3.0:1789760769.114300:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74162176 -> 74166272 00002000:00010000:3.0:1789760769.114306:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144872 -> 144880 00010000:00010000:3.0:1789760769.114321:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef71f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74162176->74166271] (req 74162176->74166271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.114341:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef71f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74162176->74166271] (req 74162176->74166271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.114350:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef71f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74162176->74166271] (req 74162176->74166271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.114357:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.114362:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef71f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74162176->74166271] (req 74162176->74166271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.114371:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef71f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74162176->74166271] (req 74162176->74166271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.114469:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef71f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74162176->74166271] (req 74162176->74166271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.114478:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef71f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74162176->74166271] (req 74162176->74166271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.114567:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503704576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6236us (7833us total) trans 4294985413 rc 0/0 00000100:00100000:3.0:1789760769.114578:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18117 00000100:00100000:3.0:1789760769.114588:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699214186676 time 699232919900 state 0x2 ref 0 00000100:00100000:0.0:1789760769.121214:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.145723:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.163816:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503704960 00000100:00100000:3.0:1789760769.163829:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503704960 00000100:00100000:3.0:1789760769.163885:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699247468824 time 699270548029 state 0x0 ref 1 00000100:00100000:3.0:1789760769.163901:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699270808445 time 699270548029 state 0x1 ref 1 00000100:00100000:3.0:1789760769.163988:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699270808445 time 699288593666 state 0x0 ref 1 00000100:00100000:3.0:1789760769.164003:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503704960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.164054:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74174464, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23600 00010000:00010000:3.0:1789760769.164074:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef734 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.164092:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.164097:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.164104:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef734 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74174464->74178559] (req 74174464->74178559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.164119:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef734 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74174464->74178559] (req 74174464->74178559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.187869:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.188061:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503705088 00000100:00100000:3.0:1789760769.188072:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503705088 00000100:00100000:0.0:1789760769.188532:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699270808445 time 699288593666 state 0x0 ref 1 00000100:00100000:0.0:1789760769.188553:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699288854082 time 699288593666 state 0x1 ref 1 00000100:00100000:0.0:1789760769.188621:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699288854082 time 699313226183 state 0x0 ref 1 00000100:00100000:0.0:1789760769.188635:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503705088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.188697:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74178560, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23608 00010000:00010000:0.0:1789760769.188706:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.188727:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760769.188733:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.188740:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.188753:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.194797:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74178560 -> 74182656 00002000:00010000:0.0:1789760769.194803:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144904 -> 144912 00010000:00010000:0.0:1789760769.194816:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.194829:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.194837:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.194844:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.194849:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.194858:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.194882:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.194889:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef73b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74178560->74182655] (req 74178560->74182655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.194978:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503705088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6342us (7116us total) trans 4294985417 rc 0/0 00000100:00100000:0.0:1789760769.194988:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18121 00000100:00100000:0.0:1789760769.194997:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699288854082 time 699313226183 state 0x2 ref 0 00000100:00100000:0.0:1789760769.205519:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.230729:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985419 is committed 00000100:00100000:0.0:1789760769.235960:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.242681:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985420 is committed 00000100:00100000:0.0:1789760769.253689:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.259013:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985421 is committed 00000100:00100000:0.0:1789760769.267814:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.272976:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985422 is committed 00000100:00100000:0.0:1789760769.279994:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.280079:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503705856 00000100:00100000:3.0:1789760769.280088:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503705856 00000100:00100000:3.0:1789760769.280236:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699378801237 time 699392682750 state 0x0 ref 1 00000100:00100000:3.0:1789760769.280253:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699392943166 time 699392682750 state 0x1 ref 1 00000100:00100000:3.0:1789760769.280290:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699392943166 time 699404896217 state 0x0 ref 1 00000100:00100000:3.0:1789760769.280303:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503705856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.280349:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74203136, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23638 00010000:00010000:3.0:1789760769.280358:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.280373:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.280377:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.280383:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.280561:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.286600:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985423 is committed 00002000:00010000:3.0:1789760769.286729:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74203136 -> 74207232 00002000:00010000:3.0:1789760769.286734:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144952 -> 144960 00010000:00010000:3.0:1789760769.286747:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.286760:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.286770:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.286784:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.286789:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.286798:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.286820:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.286829:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef765 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74203136->74207231] (req 74203136->74207231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.287026:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503705856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6719us (7036us total) trans 4294985423 rc 0/0 00000100:00100000:3.0:1789760769.287037:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18127 00000100:00100000:3.0:1789760769.287047:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699392943166 time 699404896217 state 0x2 ref 0 00000100:00100000:0.0:1789760769.293801:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.307050:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.324158:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.347358:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.347528:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503706368 00000100:00100000:3.0:1789760769.347537:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503706368 00000100:00100000:3.0:1789760769.347598:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699432112097 time 699449423468 state 0x0 ref 1 00000100:00100000:3.0:1789760769.347613:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699449683884 time 699449423468 state 0x1 ref 1 00000100:00100000:3.0:1789760769.347645:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699449683884 time 699472251648 state 0x0 ref 1 00000100:00100000:3.0:1789760769.347677:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503706368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.347734:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74219520, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23658 00010000:00010000:3.0:1789760769.347742:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.347757:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.347762:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.347768:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.347780:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.363189:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74219520 -> 74223616 00002000:00010000:3.0:1789760769.363194:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144984 -> 144992 00010000:00010000:3.0:1789760769.363208:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.363220:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.363228:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.363235:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.363240:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.363249:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.363268:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.363276:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef781 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74219520->74223615] (req 74219520->74223615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.363379:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503706368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15715us (16026us total) trans 4294985427 rc 0/0 00000100:00100000:3.0:1789760769.363435:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18131 00000100:00100000:3.0:1789760769.363446:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699449683884 time 699472251648 state 0x2 ref 0 00000100:00100000:0.0:1789760769.367071:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760769.374960:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74223616 -> 74227712 00002000:00010000:0.0:1789760769.374965:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 144992 -> 145000 00010000:00010000:0.0:1789760769.374979:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef788 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74223616->74227711] (req 74223616->74227711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.374990:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef788 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74223616->74227711] (req 74223616->74227711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.375000:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef788 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74223616->74227711] (req 74223616->74227711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.375007:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.375012:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef788 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74223616->74227711] (req 74223616->74227711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.375021:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef788 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74223616->74227711] (req 74223616->74227711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.375042:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef788 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74223616->74227711] (req 74223616->74227711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.375049:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef788 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74223616->74227711] (req 74223616->74227711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.375191:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503706496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7758us (8135us total) trans 4294985428 rc 0/0 00000100:00100000:0.0:1789760769.375201:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18132 00000100:00100000:0.0:1789760769.375211:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699472512064 time 699491985509 state 0x2 ref 0 00000100:00100000:0.0:1789760769.384023:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.412971:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503706752 00000100:00100000:3.0:1789760769.412981:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503706752 00000100:00100000:3.0:1789760769.413036:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699492245925 time 699508906355 state 0x0 ref 1 00000100:00100000:3.0:1789760769.413051:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699509166771 time 699508906355 state 0x1 ref 1 00000100:00100000:3.0:1789760769.413085:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699509166771 time 699537691644 state 0x0 ref 1 00000100:00100000:3.0:1789760769.413096:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503706752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.413142:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74231808, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23670 00010000:00010000:3.0:1789760769.413151:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.413167:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.413171:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.413177:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.413191:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.420823:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74231808 -> 74235904 00002000:00010000:3.0:1789760769.420828:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145008 -> 145016 00010000:00010000:3.0:1789760769.420840:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.420852:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.420861:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.420868:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.420873:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.420881:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.420901:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.420909:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef796 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74231808->74235903] (req 74231808->74235903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.421066:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503706752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7957us (8205us total) trans 4294985430 rc 0/0 00000100:00100000:3.0:1789760769.421076:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18134 00000100:00100000:3.0:1789760769.421086:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699509166771 time 699537691644 state 0x2 ref 0 00000100:00100000:0.0:1789760769.425271:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.425360:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503706880 00000100:00100000:3.0:1789760769.425370:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503706880 00000100:00100000:3.0:1789760769.425470:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699509166771 time 699537691644 state 0x0 ref 1 00000100:00100000:3.0:1789760769.425487:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699537952060 time 699537691644 state 0x1 ref 1 00000100:00100000:3.0:1789760769.425518:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699537952060 time 699550124589 state 0x0 ref 1 00000100:00100000:3.0:1789760769.425531:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffb6e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503706880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.425579:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74235904, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23678 00010000:00010000:3.0:1789760769.425588:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.425604:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.425609:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.425617:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.425631:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.433888:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74235904 -> 74240000 00002000:00010000:3.0:1789760769.433905:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145016 -> 145024 00010000:00010000:3.0:1789760769.433918:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.433931:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.433941:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.433948:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.433953:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.433964:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.433984:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.433999:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef79d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74235904->74239999] (req 74235904->74239999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.434117:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffb6e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503706880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8583us (8851us total) trans 4294985431 rc 0/0 00000100:00100000:3.0:1789760769.434129:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18135 00000100:00100000:3.0:1789760769.434140:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699537952060 time 699550124589 state 0x2 ref 0 00000100:00100000:0.0:1789760769.438938:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.467062:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.470786:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503707264 00000100:00100000:3.0:1789760769.470804:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503707264 00000100:00100000:3.0:1789760769.470870:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699564489060 time 699579751783 state 0x0 ref 1 00000100:00100000:3.0:1789760769.470886:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699580012199 time 699579751783 state 0x1 ref 1 00000100:00100000:3.0:1789760769.470922:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699580012199 time 699595528707 state 0x0 ref 1 00000100:00100000:3.0:1789760769.470936:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503707264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.470982:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74248192, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23690 00010000:00010000:3.0:1789760769.470991:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.471008:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.471013:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.471019:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.471038:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.478730:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74248192 -> 74252288 00002000:00010000:3.0:1789760769.478735:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145040 -> 145048 00010000:00010000:3.0:1789760769.478749:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.478763:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.478772:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.478778:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.478783:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.478792:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.478812:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.478820:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74248192->74252287] (req 74248192->74252287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.478920:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503707264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7983us (11865us total) trans 4294985434 rc 0/0 00000100:00100000:3.0:1789760769.478931:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18138 00000100:00100000:3.0:1789760769.478940:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699580012199 time 699595528707 state 0x2 ref 0 00000100:00100000:0.0:1789760769.484565:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.484653:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503707392 00000100:00100000:3.0:1789760769.484674:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503707392 00000100:00100000:3.0:1789760769.484738:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699580012199 time 699595528707 state 0x0 ref 1 00000100:00100000:3.0:1789760769.484753:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699595789123 time 699595528707 state 0x1 ref 1 00000100:00100000:3.0:1789760769.484826:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699595789123 time 699609432829 state 0x0 ref 1 00000100:00100000:3.0:1789760769.484839:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503707392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.484885:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74252288, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23698 00010000:00010000:3.0:1789760769.484894:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.484909:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.484913:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.484919:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74252288->74256383] (req 74252288->74256383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.484933:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083ef7b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74252288->74256383] (req 74252288->74256383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.515569:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74256384 -> 74260480 00002000:00010000:0.0:1789760769.515575:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145056 -> 145064 00010000:00010000:0.0:1789760769.515588:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef7c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74256384->74260479] (req 74256384->74260479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.515599:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef7c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74256384->74260479] (req 74256384->74260479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.515608:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef7c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74256384->74260479] (req 74256384->74260479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.515614:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.515619:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef7c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74256384->74260479] (req 74256384->74260479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.515627:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef7c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74256384->74260479] (req 74256384->74260479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.515645:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef7c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74256384->74260479] (req 74256384->74260479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.515657:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef7c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74256384->74260479] (req 74256384->74260479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.515761:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503707520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7115us (7319us total) trans 4294985436 rc 0/0 00000100:00100000:0.0:1789760769.515771:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18140 00000100:00100000:0.0:1789760769.515781:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699609693245 time 699633235664 state 0x2 ref 0 00000100:00100000:0.0:1789760769.528630:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.549906:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503707776 00000100:00100000:3.0:1789760769.549915:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503707776 00000100:00100000:3.0:1789760769.549976:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699633496080 time 699654605678 state 0x0 ref 1 00000100:00100000:3.0:1789760769.549990:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699654866094 time 699654605678 state 0x1 ref 1 00000100:00100000:3.0:1789760769.550017:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699654866094 time 699674624020 state 0x0 ref 1 00000100:00100000:3.0:1789760769.550028:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503707776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.550071:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74264576, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x236b0 00010000:00010000:3.0:1789760769.550080:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.550095:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.550099:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.550105:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.550120:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.556092:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760769.556136:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74264576 -> 74268672 00002000:00010000:0.0:1789760769.556142:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145072 -> 145080 00010000:00010000:0.0:1789760769.556154:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.556166:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.556175:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.556182:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.556186:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.556194:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.556212:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.556219:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083ef7ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74264576->74268671] (req 74264576->74268671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.556556:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503707776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6522us (6766us total) trans 4294985438 rc 0/0 00000100:00100000:0.0:1789760769.556567:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18142 00000100:00100000:0.0:1789760769.556577:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699654866094 time 699674624020 state 0x2 ref 0 00000100:00100000:0.0:1789760769.562594:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.573442:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503708032 00000100:00100000:0.0:1789760769.573453:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503708032 00000100:00100000:0.0:1789760769.573510:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699674884436 time 699687707068 state 0x0 ref 1 00000100:00100000:0.0:1789760769.573527:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699687967484 time 699687707068 state 0x1 ref 1 00000100:00100000:0.0:1789760769.573562:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699687967484 time 699698168859 state 0x0 ref 1 00000100:00100000:0.0:1789760769.573575:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.573622:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74272768, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x236c0 00010000:00010000:0.0:1789760769.573631:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.573652:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760769.573668:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.573675:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.573690:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.578216:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74272768 -> 74276864 00002000:00010000:0.0:1789760769.578222:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145088 -> 145096 00010000:00010000:0.0:1789760769.578236:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.578249:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.578258:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.578266:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.578271:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.578280:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.578306:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.578315:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74272768->74276863] (req 74272768->74276863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.578652:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5075us (5349us total) trans 4294985440 rc 0/0 00000100:00100000:0.0:1789760769.578674:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18144 00000100:00100000:0.0:1789760769.578684:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699687967484 time 699698168859 state 0x2 ref 0 00000100:00100000:0.0:1789760769.582959:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503708160 00000100:00100000:0.0:1789760769.582968:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503708160 00000100:00100000:0.0:1789760769.583023:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699687967484 time 699698168859 state 0x0 ref 1 00000100:00100000:0.0:1789760769.583037:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699698429275 time 699698168859 state 0x1 ref 1 00000100:00100000:0.0:1789760769.583070:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699698429275 time 699707675955 state 0x0 ref 1 00000100:00100000:0.0:1789760769.583083:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.583131:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74276864, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x236c8 00010000:00010000:0.0:1789760769.583141:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.583156:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760769.583162:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.583168:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.583182:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.587844:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74276864 -> 74280960 00002000:00010000:0.0:1789760769.587849:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145096 -> 145104 00010000:00010000:0.0:1789760769.587863:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.587876:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.587885:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.587892:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.587897:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.587906:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.587927:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.587940:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74276864->74280959] (req 74276864->74280959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.588077:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4991us (5208us total) trans 4294985441 rc 0/0 00000100:00100000:0.0:1789760769.588088:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18145 00000100:00100000:0.0:1789760769.588097:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699698429275 time 699707675955 state 0x2 ref 0 00000100:00100000:3.0:1789760769.592078:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503708288 00000100:00100000:3.0:1789760769.592226:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503708288 00000100:00100000:3.0:1789760769.592277:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699698429275 time 699707675955 state 0x0 ref 1 00000100:00100000:3.0:1789760769.592294:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699707936371 time 699707675955 state 0x1 ref 1 00000100:00100000:3.0:1789760769.592349:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699707936371 time 699716955701 state 0x0 ref 1 00000100:00100000:3.0:1789760769.592361:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.601905:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74280960 -> 74285056 00002000:00010000:0.0:1789760769.601911:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145104 -> 145112 00010000:00010000:0.0:1789760769.601924:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef7ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74280960->74285055] (req 74280960->74285055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.601936:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef7ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74280960->74285055] (req 74280960->74285055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.601944:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef7ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74280960->74285055] (req 74280960->74285055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.601951:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.601955:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef7ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74280960->74285055] (req 74280960->74285055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.601963:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef7ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74280960->74285055] (req 74280960->74285055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.601988:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef7ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74280960->74285055] (req 74280960->74285055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.601996:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083ef7ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74280960->74285055] (req 74280960->74285055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.602083:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9719us (10257us total) trans 4294985442 rc 0/0 00000100:00100000:0.0:1789760769.602093:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18146 00000100:00100000:0.0:1789760769.602103:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699707936371 time 699716955701 state 0x2 ref 0 00000100:00100000:0.0:1789760769.606290:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503708416 00000100:00100000:0.0:1789760769.606301:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503708416 00000100:00100000:0.0:1789760769.606355:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699707936371 time 699716955701 state 0x0 ref 1 00000100:00100000:0.0:1789760769.606369:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699717216117 time 699716955701 state 0x1 ref 1 00000100:00100000:0.0:1789760769.606420:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699717216117 time 699731025916 state 0x0 ref 1 00000100:00100000:0.0:1789760769.606434:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.606482:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74285056, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x236d8 00010000:00010000:0.0:1789760769.606492:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.606508:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760769.606515:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.606523:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.606537:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.611324:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74285056 -> 74289152 00002000:00010000:0.0:1789760769.611330:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145112 -> 145120 00010000:00010000:0.0:1789760769.611342:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.611354:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.611364:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.611372:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.611377:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.611432:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.611451:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.611464:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef7f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74285056->74289151] (req 74285056->74289151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.611544:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5110us (5354us total) trans 4294985443 rc 0/0 00000100:00100000:0.0:1789760769.611554:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18147 00000100:00100000:0.0:1789760769.611564:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699717216117 time 699731025916 state 0x2 ref 0 00000100:00100000:0.0:1789760769.615289:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503708544 00000100:00100000:0.0:1789760769.615299:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503708544 00000100:00100000:0.0:1789760769.615366:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699717216117 time 699731025916 state 0x0 ref 1 00000100:00100000:0.0:1789760769.615574:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699731286332 time 699731025916 state 0x1 ref 1 00000100:00100000:0.0:1789760769.615610:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699731286332 time 699740216469 state 0x0 ref 1 00000100:00100000:0.0:1789760769.615622:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.615678:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74289152, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x236e0 00010000:00010000:0.0:1789760769.615688:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.615703:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760769.615708:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.615715:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.615736:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.621288:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74289152 -> 74293248 00002000:00010000:0.0:1789760769.621293:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145120 -> 145128 00010000:00010000:0.0:1789760769.621305:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.621318:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.621336:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.621344:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.621349:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.621358:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.621374:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.621426:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef7f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74289152->74293247] (req 74289152->74293247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.638108:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503708800 00000100:00100000:0.0:1789760769.638117:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503708800 00000100:00100000:0.0:1789760769.638164:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699740476885 time 699754321197 state 0x0 ref 1 00000100:00100000:0.0:1789760769.638179:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699754581613 time 699754321197 state 0x1 ref 1 00000100:00100000:0.0:1789760769.638207:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699754581613 time 699762814329 state 0x0 ref 1 00000100:00100000:0.0:1789760769.638217:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.638257:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74297344, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x236f0 00010000:00010000:0.0:1789760769.638266:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.638280:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760769.638284:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.638290:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.638302:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.642591:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74297344 -> 74301440 00002000:00010000:0.0:1789760769.642597:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145136 -> 145144 00010000:00010000:0.0:1789760769.642611:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642630:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642640:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642648:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.642653:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642673:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642694:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642884:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083ef806 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74297344->74301439] (req 74297344->74301439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.643235:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503708800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5013us (5194us total) trans 4294985446 rc 0/0 00000100:00100000:0.0:1789760769.643247:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18150 00000100:00100000:0.0:1789760769.643257:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699754581613 time 699762814329 state 0x2 ref 0 00000100:00100000:0.0:1789760769.646925:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503708928 00000100:00100000:0.0:1789760769.646936:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503708928 00000100:00100000:0.0:1789760769.646988:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699754581613 time 699762814329 state 0x0 ref 1 00000100:00100000:0.0:1789760769.647074:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699763074745 time 699762814329 state 0x1 ref 1 00000100:00100000:0.0:1789760769.647105:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699763074745 time 699771712202 state 0x0 ref 1 00000100:00100000:0.0:1789760769.647118:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503708928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.647164:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74301440, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x236f8 00010000:00010000:0.0:1789760769.647172:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.647186:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760769.647191:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.647198:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.647211:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.650903:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74301440 -> 74305536 00002000:00010000:0.0:1789760769.650908:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145144 -> 145152 00010000:00010000:0.0:1789760769.650920:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.650932:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.650947:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.650954:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.650959:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.650968:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.650986:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.650994:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef80d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74301440->74305535] (req 74301440->74305535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.651208:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503708928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4087us (4386us total) trans 4294985447 rc 0/0 00000100:00100000:0.0:1789760769.651219:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18151 00000100:00100000:0.0:1789760769.651229:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699763074745 time 699771712202 state 0x2 ref 0 00000100:00100000:0.0:1789760769.663584:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503709184 00000100:00100000:0.0:1789760769.663595:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503709184 00000100:00100000:0.0:1789760769.663649:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699771972618 time 699780442784 state 0x0 ref 1 00000100:00100000:0.0:1789760769.663676:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699780703200 time 699780442784 state 0x1 ref 1 00000100:00100000:0.0:1789760769.671010:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.680272:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.695910:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.695993:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503709568 00000100:00100000:3.0:1789760769.696003:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503709568 00000100:00100000:3.0:1789760769.696057:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699796112404 time 699805070184 state 0x0 ref 1 00000100:00100000:3.0:1789760769.696133:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699805330600 time 699805070184 state 0x1 ref 1 00000100:00100000:3.0:1789760769.696164:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699805330600 time 699820770222 state 0x0 ref 1 00000100:00100000:3.0:1789760769.696176:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503709568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.696224:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74321920, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23720 00010000:00010000:3.0:1789760769.696234:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.696249:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.696253:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.696259:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.696273:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.702132:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74321920 -> 74326016 00002000:00010000:3.0:1789760769.702138:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145184 -> 145192 00010000:00010000:3.0:1789760769.702150:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.702161:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.702169:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.702177:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.702181:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.702190:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.702209:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.702215:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef830 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74321920->74326015] (req 74321920->74326015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.702344:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503709568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6164us (6437us total) trans 4294985452 rc 0/0 00000100:00100000:3.0:1789760769.702355:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18156 00000100:00100000:3.0:1789760769.702365:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699805330600 time 699820770222 state 0x2 ref 0 00000100:00100000:0.0:1789760769.707125:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.719158:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760769.729618:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985454 is committed 00000100:00100000:0.0:1789760769.739553:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760769.747145:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985455 is committed 00000100:00100000:0.0:1789760769.754896:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.755064:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503710080 00000100:00100000:3.0:1789760769.755074:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503710080 00000100:00100000:3.0:1789760769.755127:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699844849461 time 699865888543 state 0x0 ref 1 00000100:00100000:3.0:1789760769.755142:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699866148959 time 699865888543 state 0x1 ref 1 00000100:00100000:3.0:1789760769.755212:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699866148959 time 699879817185 state 0x0 ref 1 00000100:00100000:3.0:1789760769.755225:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503710080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.755273:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74338304, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x23740 00010000:00010000:3.0:1789760769.755282:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.755308:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.755313:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.755319:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.755334:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.762022:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74338304 -> 74342400 00002000:00010000:3.0:1789760769.762027:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145216 -> 145224 00010000:00010000:3.0:1789760769.762040:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.762052:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.762062:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.762069:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.762074:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.762084:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.762102:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.762110:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef84c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74338304->74342399] (req 74338304->74342399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.762247:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503710080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7019us (7422us total) trans 4294985456 rc 0/0 00000100:00100000:3.0:1789760769.762259:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18160 00000100:00100000:3.0:1789760769.762269:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699866148959 time 699879817185 state 0x2 ref 0 00000100:00100000:0.0:1789760769.773164:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760769.785968:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74342400 -> 74346496 00002000:00010000:0.0:1789760769.785974:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145224 -> 145232 00010000:00010000:0.0:1789760769.785987:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef853 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74342400->74346495] (req 74342400->74346495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.786010:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef853 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74342400->74346495] (req 74342400->74346495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.786020:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef853 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74342400->74346495] (req 74342400->74346495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.786027:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.786032:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef853 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74342400->74346495] (req 74342400->74346495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.786041:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef853 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74342400->74346495] (req 74342400->74346495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.786060:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef853 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74342400->74346495] (req 74342400->74346495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.786070:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083ef853 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74342400->74346495] (req 74342400->74346495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.786160:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503710208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12687us (13001us total) trans 4294985457 rc 0/0 00000100:00100000:0.0:1789760769.786170:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18161 00000100:00100000:0.0:1789760769.786181:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699880077601 time 699897974310 state 0x2 ref 0 00000100:00100000:0.0:1789760769.791920:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760769.800208:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985458 is committed 00000001:00080000:0.0:1789760769.812288:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985459 is committed 00002000:00010000:3.0:1789760769.812522:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74350592 -> 74354688 00002000:00010000:3.0:1789760769.812528:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145240 -> 145248 00010000:00010000:3.0:1789760769.812542:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef861 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74350592->74354687] (req 74350592->74354687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.812555:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef861 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74350592->74354687] (req 74350592->74354687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.812564:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef861 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74350592->74354687] (req 74350592->74354687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.812572:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.812576:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef861 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74350592->74354687] (req 74350592->74354687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.812943:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef861 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74350592->74354687] (req 74350592->74354687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.812967:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef861 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74350592->74354687] (req 74350592->74354687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.812975:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083ef861 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74350592->74354687] (req 74350592->74354687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.813071:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503710464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6635us (6873us total) trans 4294985459 rc 0/0 00000100:00100000:3.0:1789760769.813081:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18163 00000100:00100000:3.0:1789760769.813090:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699917846854 time 699930997173 state 0x2 ref 0 00000100:00100000:0.0:1789760769.815994:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.816195:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503710592 00000100:00100000:3.0:1789760769.816205:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503710592 00000100:00100000:3.0:1789760769.816258:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699917846854 time 699930997173 state 0x0 ref 1 00000100:00100000:3.0:1789760769.816275:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699931257589 time 699930997173 state 0x1 ref 1 00000100:00100000:3.0:1789760769.816434:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699931257589 time 699941039151 state 0x0 ref 1 00000100:00100000:3.0:1789760769.816448:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503710592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.816497:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74354688, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23760 00010000:00010000:3.0:1789760769.816506:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.816522:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.816526:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.816533:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.816547:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.822786:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985460 is committed 00002000:00010000:3.0:1789760769.822905:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74354688 -> 74358784 00002000:00010000:3.0:1789760769.822910:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145248 -> 145256 00010000:00010000:3.0:1789760769.822923:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.822935:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.822944:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.822951:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.822956:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.822966:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.822989:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.822997:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef868 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74354688->74358783] (req 74354688->74358783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.823092:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503710592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6641us (7103us total) trans 4294985460 rc 0/0 00000100:00100000:3.0:1789760769.823102:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18164 00000100:00100000:3.0:1789760769.823113:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699931257589 time 699941039151 state 0x2 ref 0 00000100:00100000:0.0:1789760769.832004:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.857667:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.857879:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503710976 00000100:00100000:3.0:1789760769.857889:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503710976 00000100:00100000:3.0:1789760769.857941:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699957299640 time 699971178045 state 0x0 ref 1 00000100:00100000:3.0:1789760769.857957:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699971438461 time 699971178045 state 0x1 ref 1 00000100:00100000:3.0:1789760769.858033:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699971438461 time 699982639362 state 0x0 ref 1 00000100:00100000:3.0:1789760769.858047:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503710976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.858092:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74366976, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23778 00010000:00010000:3.0:1789760769.858102:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.858119:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.858124:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.858130:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.858143:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.863990:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74366976 -> 74371072 00002000:00010000:3.0:1789760769.863996:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145272 -> 145280 00010000:00010000:3.0:1789760769.864010:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.864023:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.864031:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.864039:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.864044:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.864053:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.864073:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.864081:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef87d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74366976->74371071] (req 74366976->74371071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.864201:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503710976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6151us (6547us total) trans 4294985463 rc 0/0 00000100:00100000:3.0:1789760769.864212:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18167 00000100:00100000:3.0:1789760769.864238:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699971438461 time 699982639362 state 0x2 ref 0 00000100:00100000:0.0:1789760769.869220:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.869357:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503711104 00000100:00100000:3.0:1789760769.869367:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503711104 00000100:00100000:3.0:1789760769.869466:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699971438461 time 699982639362 state 0x0 ref 1 00000100:00100000:3.0:1789760769.869481:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699982899778 time 699982639362 state 0x1 ref 1 00000100:00100000:3.0:1789760769.869516:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699982899778 time 699994122698 state 0x0 ref 1 00000100:00100000:3.0:1789760769.869529:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503711104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.869572:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74371072, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23780 00010000:00010000:3.0:1789760769.869581:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869596:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760769.869601:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.869607:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869620:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.874951:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985464 is committed 00002000:00010000:3.0:1789760769.875071:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74371072 -> 74375168 00002000:00010000:3.0:1789760769.875077:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145280 -> 145288 00010000:00010000:3.0:1789760769.875090:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.875109:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.875118:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.875126:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.875131:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.875140:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.875164:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.875172:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef884 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74371072->74375167] (req 74371072->74375167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.875952:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503711104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6418us (6734us total) trans 4294985464 rc 0/0 00000100:00100000:3.0:1789760769.875963:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18168 00000100:00100000:3.0:1789760769.875972:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699982899778 time 699994122698 state 0x2 ref 0 00000100:00100000:0.0:1789760769.880322:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503711232 00000100:00100000:0.0:1789760769.880331:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503711232 00000100:00100000:0.0:1789760769.880774:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699982899778 time 699994122698 state 0x0 ref 1 00000100:00100000:0.0:1789760769.880792:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699994383114 time 699994122698 state 0x1 ref 1 00000100:00100000:0.0:1789760769.880860:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699994383114 time 700005465295 state 0x0 ref 1 00000100:00100000:0.0:1789760769.880875:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503711232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.880925:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74375168, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23788 00010000:00010000:0.0:1789760769.880934:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.880949:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760769.880954:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.880961:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.880978:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.886490:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985465 is committed 00002000:00010000:0.0:1789760769.886615:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74375168 -> 74379264 00002000:00010000:0.0:1789760769.886621:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145288 -> 145296 00010000:00010000:0.0:1789760769.886634:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.886647:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.886664:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.886673:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.886678:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.886692:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.886712:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.886721:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083ef88b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74375168->74379263] (req 74375168->74379263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.886815:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503711232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5940us (6642us total) trans 4294985465 rc 0/0 00000100:00100000:0.0:1789760769.886825:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18169 00000100:00100000:0.0:1789760769.886834:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 699994383114 time 700005465295 state 0x2 ref 0 00000100:00100000:0.0:1789760769.891858:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760769.898648:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74379264 -> 74383360 00002000:00010000:0.0:1789760769.898654:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145296 -> 145304 00010000:00010000:0.0:1789760769.898677:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef892 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74379264->74383359] (req 74379264->74383359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.898689:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef892 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74379264->74383359] (req 74379264->74383359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.898767:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef892 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74379264->74383359] (req 74379264->74383359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.898781:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.898787:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef892 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74379264->74383359] (req 74379264->74383359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.898797:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef892 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74379264->74383359] (req 74379264->74383359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.898819:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef892 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74379264->74383359] (req 74379264->74383359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.898826:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083ef892 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74379264->74383359] (req 74379264->74383359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.898973:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503711360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6517us (7120us total) trans 4294985466 rc 0/0 00000100:00100000:0.0:1789760769.898985:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18170 00000100:00100000:0.0:1789760769.898996:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700005725711 time 700017043105 state 0x2 ref 0 00000100:00100000:3.0:1789760769.902768:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503711488 00000100:00100000:3.0:1789760769.902778:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503711488 00000100:00100000:3.0:1789760769.902835:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700005725711 time 700017043105 state 0x0 ref 1 00000100:00100000:3.0:1789760769.902851:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700017303521 time 700017043105 state 0x1 ref 1 00000100:00100000:3.0:1789760769.902914:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700017303521 time 700027520607 state 0x0 ref 1 00000100:00100000:3.0:1789760769.902927:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503711488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.902972:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74383360, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23798 00010000:00010000:3.0:1789760769.902981:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef899 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.902996:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760769.903001:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.903008:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef899 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74383360->74387455] (req 74383360->74387455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.903021:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083ef899 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74383360->74387455] (req 74383360->74387455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.918478:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503711616 00000100:00100000:3.0:1789760769.918488:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503711616 00000100:00100000:3.0:1789760769.918543:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700017303521 time 700027520607 state 0x0 ref 1 00000100:00100000:3.0:1789760769.918613:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700027781023 time 700027520607 state 0x1 ref 1 00000100:00100000:3.0:1789760769.918647:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700027781023 time 700043253337 state 0x0 ref 1 00000100:00100000:3.0:1789760769.918670:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503711616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.918718:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74387456, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x237a0 00010000:00010000:3.0:1789760769.918727:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.918744:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760769.918749:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.918755:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.918769:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.922924:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74387456 -> 74391552 00002000:00010000:3.0:1789760769.922929:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145312 -> 145320 00010000:00010000:3.0:1789760769.922957:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.922970:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.922979:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.922995:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760769.923000:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.923010:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.923030:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.923037:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74387456->74391551] (req 74387456->74391551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.923163:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503711616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4501us (4834us total) trans 4294985468 rc 0/0 00000100:00100000:3.0:1789760769.923174:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18172 00000100:00100000:3.0:1789760769.923183:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700027781023 time 700043253337 state 0x2 ref 0 00000100:00100000:0.0:1789760769.928962:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760769.947334:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503711872 00000100:00100000:0.0:1789760769.947345:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503711872 00000100:00100000:0.0:1789760769.947447:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700043513753 time 700053845823 state 0x0 ref 1 00000100:00100000:0.0:1789760769.947464:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700054106239 time 700053845823 state 0x1 ref 1 00000100:00100000:0.0:1789760769.947488:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700054106239 time 700072095227 state 0x0 ref 1 00000100:00100000:0.0:1789760769.947504:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503711872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760769.947562:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74395648, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x237b0 00010000:00010000:0.0:1789760769.947573:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083ef8ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.947589:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760769.947594:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.947601:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083ef8ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74395648->74399743] (req 74395648->74399743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.947616:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083ef8ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74395648->74399743] (req 74395648->74399743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.954927:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985470 is committed 00000100:00100000:0.0:1789760769.960441:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760769.968090:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985471 is committed 00000100:00100000:0.0:1789760769.976896:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760769.976980:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503712128 00000100:00100000:3.0:1789760769.977000:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503712128 00000100:00100000:3.0:1789760769.977057:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700072355643 time 700087298724 state 0x0 ref 1 00000100:00100000:3.0:1789760769.977069:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700087559140 time 700087298724 state 0x1 ref 1 00000100:00100000:3.0:1789760769.977094:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700087559140 time 700101700324 state 0x0 ref 1 00000100:00100000:3.0:1789760769.977105:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503712128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760769.977159:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74403840, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x237c0 00010000:00010000:3.0:1789760769.977169:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.977185:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760769.977189:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.977196:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.977209:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.987115:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74403840 -> 74407936 00002000:00010000:0.0:1789760769.987122:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145344 -> 145352 00010000:00010000:0.0:1789760769.987135:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.987147:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.987156:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.987165:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760769.987169:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.987179:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.987200:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.987207:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083ef8bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74403840->74407935] (req 74403840->74407935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.987449:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503712128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10338us (10557us total) trans 4294985472 rc 0/0 00000100:00100000:0.0:1789760769.987461:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18176 00000100:00100000:0.0:1789760769.987471:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700087559140 time 700101700324 state 0x2 ref 0 00000100:00100000:0.0:1789760770.000462:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503712256 00000100:00100000:0.0:1789760770.000477:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503712256 00000100:00100000:0.0:1789760770.000559:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700087559140 time 700101700324 state 0x0 ref 1 00000100:00100000:0.0:1789760770.000572:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700101960740 time 700101700324 state 0x1 ref 1 00000100:00100000:0.0:1789760770.000596:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700101960740 time 700125203002 state 0x0 ref 1 00000100:00100000:0.0:1789760770.000614:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503712256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.000669:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74407936, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x237c8 00010000:00010000:0.0:1789760770.000679:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef8c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.000695:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.000700:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.000706:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef8c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74407936->74412031] (req 74407936->74412031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.000721:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083ef8c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74407936->74412031] (req 74407936->74412031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.023574:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985474 is committed 00000001:00080000:0.0:1789760770.035851:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985475 is committed 00000100:00100000:0.0:1789760770.040618:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503712640 00000100:00100000:0.0:1789760770.040628:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503712640 00000100:00100000:0.0:1789760770.040691:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700138712071 time 700152835960 state 0x0 ref 1 00000100:00100000:0.0:1789760770.040706:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700153096376 time 700152835960 state 0x1 ref 1 00000100:00100000:0.0:1789760770.040731:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700153096376 time 700165338334 state 0x0 ref 1 00000100:00100000:0.0:1789760770.040744:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503712640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.040795:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74420224, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x237e0 00010000:00010000:0.0:1789760770.040805:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.040821:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.040825:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.040831:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.040845:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.045971:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74420224 -> 74424320 00002000:00010000:0.0:1789760770.045977:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145376 -> 145384 00010000:00010000:0.0:1789760770.045997:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.046010:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.046019:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.046027:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.046032:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.046042:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.046062:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.046071:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74420224->74424319] (req 74420224->74424319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.046165:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503712640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5418us (5648us total) trans 4294985476 rc 0/0 00000100:00100000:0.0:1789760770.046176:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18180 00000100:00100000:0.0:1789760770.046186:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700153096376 time 700165338334 state 0x2 ref 0 00000001:00080000:0.0:1789760770.056273:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985477 is committed 00002000:00010000:0.0:1789760770.074192:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74428416 -> 74432512 00002000:00010000:0.0:1789760770.074198:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145392 -> 145400 00010000:00010000:0.0:1789760770.074212:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef8e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74428416->74432511] (req 74428416->74432511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.074227:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef8e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74428416->74432511] (req 74428416->74432511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.074237:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef8e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74428416->74432511] (req 74428416->74432511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.074244:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.074248:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef8e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74428416->74432511] (req 74428416->74432511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.074258:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef8e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74428416->74432511] (req 74428416->74432511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.074276:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef8e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74428416->74432511] (req 74428416->74432511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.074283:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083ef8e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74428416->74432511] (req 74428416->74432511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.074453:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503712896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10204us (11312us total) trans 4294985478 rc 0/0 00000100:00100000:0.0:1789760770.074464:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18182 00000100:00100000:0.0:1789760770.074474:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700176431816 time 700188825544 state 0x2 ref 0 00000001:00080000:0.0:1789760770.091324:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985479 is committed 00000100:00100000:0.0:1789760770.097701:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503713152 00000100:00100000:0.0:1789760770.097712:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503713152 00000100:00100000:0.0:1789760770.097764:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700189085960 time 700209934963 state 0x0 ref 1 00000100:00100000:0.0:1789760770.097779:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700210195379 time 700209934963 state 0x1 ref 1 00000100:00100000:0.0:1789760770.097813:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700210195379 time 700222419368 state 0x0 ref 1 00000100:00100000:0.0:1789760770.097825:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503713152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.097873:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74436608, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23800 00010000:00010000:0.0:1789760770.097883:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.097898:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.097903:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.097909:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.097923:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.105848:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74436608 -> 74440704 00002000:00010000:0.0:1789760770.105854:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145408 -> 145416 00010000:00010000:0.0:1789760770.105868:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.105880:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.105889:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.105896:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.105901:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.105910:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.106113:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.106124:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef8f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74436608->74440703] (req 74436608->74440703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.106264:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503713152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8436us (8765us total) trans 4294985480 rc 0/0 00000100:00100000:0.0:1789760770.106275:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18184 00000100:00100000:0.0:1789760770.106285:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700210195379 time 700222419368 state 0x2 ref 0 00000001:00080000:0.0:1789760770.127871:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985481 is committed 00000001:00080000:0.0:1789760770.145325:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985482 is committed 00000001:00080000:0.0:1789760770.162948:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985483 is committed 00002000:00010000:3.0:1789760770.163141:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74448896 -> 74452992 00002000:00010000:3.0:1789760770.163146:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145432 -> 145440 00010000:00010000:3.0:1789760770.163158:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ef909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74448896->74452991] (req 74448896->74452991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.163169:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ef909 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74448896->74452991] (req 74448896->74452991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.163178:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ef909 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74448896->74452991] (req 74448896->74452991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.163186:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760770.163191:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ef909 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74448896->74452991] (req 74448896->74452991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.163201:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ef909 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74448896->74452991] (req 74448896->74452991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.163220:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ef909 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74448896->74452991] (req 74448896->74452991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.163228:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083ef909 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74448896->74452991] (req 74448896->74452991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.163370:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503713536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10842us (11232us total) trans 4294985483 rc 0/0 00000100:00100000:3.0:1789760770.163382:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18187 00000100:00100000:3.0:1789760770.163433:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700264050961 time 700277119443 state 0x2 ref 0 00000100:00100000:0.0:1789760770.174710:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503713664 00000100:00100000:0.0:1789760770.174721:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503713664 00000100:00100000:0.0:1789760770.174777:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700264050961 time 700277119443 state 0x0 ref 1 00000100:00100000:0.0:1789760770.174793:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700277379859 time 700277119443 state 0x1 ref 1 00000100:00100000:0.0:1789760770.174830:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700277379859 time 700299435912 state 0x0 ref 1 00000100:00100000:0.0:1789760770.174844:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503713664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.174893:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74452992, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23820 00010000:00010000:0.0:1789760770.174903:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.174922:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.174927:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.174935:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.174950:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.184452:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74452992 -> 74457088 00002000:00010000:0.0:1789760770.184458:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145440 -> 145448 00010000:00010000:0.0:1789760770.184471:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.184484:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.184494:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.184502:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.184506:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.184516:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.184542:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.184550:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef910 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74452992->74457087] (req 74452992->74457087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.184647:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503713664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9801us (10050us total) trans 4294985484 rc 0/0 00000100:00100000:0.0:1789760770.184666:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18188 00000100:00100000:0.0:1789760770.184676:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700277379859 time 700299435912 state 0x2 ref 0 00000001:00080000:0.0:1789760770.200950:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985485 is committed 00000001:00080000:0.0:1789760770.220840:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985486 is committed 00000100:00100000:3.0:1789760770.233256:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503714048 00000100:00100000:3.0:1789760770.233266:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503714048 00000100:00100000:3.0:1789760770.233324:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700313717613 time 700332498976 state 0x0 ref 1 00000100:00100000:3.0:1789760770.233339:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700332759392 time 700332498976 state 0x1 ref 1 00000100:00100000:3.0:1789760770.233475:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700332759392 time 700358079747 state 0x0 ref 1 00000100:00100000:3.0:1789760770.233490:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503714048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.233540:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74465280, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23838 00010000:00010000:3.0:1789760770.233550:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.233564:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.233569:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.233574:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.233588:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.240638:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985487 is committed 00002000:00010000:3.0:1789760770.240827:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74465280 -> 74469376 00002000:00010000:3.0:1789760770.240832:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145464 -> 145472 00010000:00010000:3.0:1789760770.240847:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240858:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240866:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240873:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760770.240878:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240886:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240906:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240919:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef925 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74465280->74469375] (req 74465280->74469375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.241013:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503714048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7520us (8315us total) trans 4294985487 rc 0/0 00000100:00100000:3.0:1789760770.241023:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18191 00000100:00100000:3.0:1789760770.241033:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700332759392 time 700358079747 state 0x2 ref 0 00000100:00100000:0.0:1789760770.246840:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503714176 00000100:00100000:0.0:1789760770.246851:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503714176 00000100:00100000:0.0:1789760770.246914:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700332759392 time 700358079747 state 0x0 ref 1 00000100:00100000:0.0:1789760770.246932:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700358340163 time 700358079747 state 0x1 ref 1 00000100:00100000:0.0:1789760770.246965:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700358340163 time 700371571302 state 0x0 ref 1 00000100:00100000:0.0:1789760770.246978:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffb6f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503714176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.247025:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74469376, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23840 00010000:00010000:0.0:1789760770.247035:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.247051:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.247055:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.247061:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.247076:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.253129:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74469376 -> 74473472 00002000:00010000:0.0:1789760770.253135:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145472 -> 145480 00010000:00010000:0.0:1789760770.253149:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.253161:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.253170:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.253178:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.253182:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.253191:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.253217:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.253225:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef92c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74469376->74473471] (req 74469376->74473471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.253321:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffb6f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503714176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6341us (6578us total) trans 4294985488 rc 0/0 00000100:00100000:0.0:1789760770.253331:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18192 00000100:00100000:0.0:1789760770.253341:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700358340163 time 700371571302 state 0x2 ref 0 00000100:00100000:0.0:1789760770.258896:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cce83d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523605632:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760770.258982:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760770.259432:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523605760:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760770.259464:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760770.260063:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523605760:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760770.260127:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cce83d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523605632:0@lo:13:osp-pre-0-1.0 00000001:00080000:0.0:1789760770.264017:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985489 is committed 00000001:00080000:0.0:1789760770.273672:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985490 is committed 00000100:00100000:3.0:1789760770.277475:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503714560 00000100:00100000:3.0:1789760770.277486:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503714560 00000100:00100000:3.0:1789760770.277540:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700382073244 time 700392951984 state 0x0 ref 1 00000100:00100000:3.0:1789760770.277555:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700393212400 time 700392951984 state 0x1 ref 1 00000100:00100000:3.0:1789760770.277579:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700393212400 time 700402186014 state 0x0 ref 1 00000100:00100000:3.0:1789760770.277592:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503714560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.277637:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74481664, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23858 00010000:00010000:3.0:1789760770.277647:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef941 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.277679:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.277684:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.278956:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74481664->74485759] (req 74481664->74485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.278983:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083ef941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74481664->74485759] (req 74481664->74485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.288643:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503714688 00000100:00100000:0.0:1789760770.288661:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503714688 00000100:00100000:0.0:1789760770.288834:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700393212400 time 700402186014 state 0x0 ref 1 00000100:00100000:0.0:1789760770.288849:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700402446430 time 700402186014 state 0x1 ref 1 00000100:00100000:0.0:1789760770.313709:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503714944 00000100:00100000:0.0:1789760770.313719:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503714944 00000100:00100000:0.0:1789760770.313773:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700413777520 time 700426461199 state 0x0 ref 1 00000100:00100000:0.0:1789760770.313957:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700426721615 time 700426461199 state 0x1 ref 1 00000100:00100000:0.0:1789760770.313986:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700426721615 time 700438592693 state 0x0 ref 1 00000100:00100000:0.0:1789760770.313999:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503714944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.314045:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74493952, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23870 00010000:00010000:0.0:1789760770.314055:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.314069:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.314073:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.314078:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.314194:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.324539:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74493952 -> 74498048 00002000:00010000:0.0:1789760770.324553:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145520 -> 145528 00010000:00010000:0.0:1789760770.324566:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324579:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324588:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324596:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.324601:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324611:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324631:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324639:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef956 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74493952->74498047] (req 74493952->74498047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.324903:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503714944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10901us (11360us total) trans 4294985494 rc 0/0 00000100:00100000:0.0:1789760770.324913:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18198 00000100:00100000:0.0:1789760770.324923:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700426721615 time 700438592693 state 0x2 ref 0 00000100:00100000:0.0:1789760770.342823:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503715200 00000100:00100000:0.0:1789760770.342834:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503715200 00000100:00100000:0.0:1789760770.342890:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700438853109 time 700454754447 state 0x0 ref 1 00000100:00100000:0.0:1789760770.342905:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700455014863 time 700454754447 state 0x1 ref 1 00000100:00100000:0.0:1789760770.342938:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700455014863 time 700467544525 state 0x0 ref 1 00000100:00100000:0.0:1789760770.342953:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503715200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.343002:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74502144, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23880 00010000:00010000:0.0:1789760770.343011:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.343027:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.343031:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.343037:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.343051:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.348844:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74502144 -> 74506240 00002000:00010000:0.0:1789760770.348850:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145536 -> 145544 00010000:00010000:0.0:1789760770.348863:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.348876:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.348886:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.348894:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.348899:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.348908:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.348929:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.348937:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083ef964 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74502144->74506239] (req 74502144->74506239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.349070:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503715200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6114us (6339us total) trans 4294985496 rc 0/0 00000100:00100000:0.0:1789760770.349081:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18200 00000100:00100000:0.0:1789760770.349091:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700455014863 time 700467544525 state 0x2 ref 0 00000001:00080000:0.0:1789760770.373781:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985498 is committed 00000100:00100000:0.0:1789760770.393217:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503715712 00000100:00100000:0.0:1789760770.398261:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503715712 00000100:00100000:3.0:1789760770.400691:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503715712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.400759:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74518528, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x238a0 00010000:00010000:3.0:1789760770.400768:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400784:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.400789:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.400796:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400814:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.416544:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74518528 -> 74522624 00002000:00010000:3.0:1789760770.416554:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145568 -> 145576 00010000:00010000:3.0:1789760770.416566:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.416584:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.416593:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.416599:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760770.416603:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.416612:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.416628:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.416635:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef980 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74518528->74522623] (req 74518528->74522623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.416892:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503715712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16281us (23934us total) trans 4294985500 rc 0/0 00000100:00100000:3.0:1789760770.416902:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18204 00000100:00100000:3.0:1789760770.416910:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700503070733 time 700523254185 state 0x2 ref 0 00000001:00080000:3.0:1789760770.449873:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985502 is committed 00000001:00080000:3.0:1789760770.463139:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985503 is committed 00000100:00100000:0.0:1789760770.467149:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760770.467231:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503716224 00000100:00100000:3.0:1789760770.467241:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503716224 00000100:00100000:3.0:1789760770.467295:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700563770463 time 700579786613 state 0x0 ref 1 00000100:00100000:3.0:1789760770.467310:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700580047029 time 700579786613 state 0x1 ref 1 00000100:00100000:3.0:1789760770.467347:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700580047029 time 700591954004 state 0x0 ref 1 00000100:00100000:3.0:1789760770.467360:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503716224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.467440:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74534912, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x238c0 00010000:00010000:3.0:1789760770.467449:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.467466:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.467470:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.467477:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.467491:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.475321:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74534912 -> 74539008 00002000:00010000:3.0:1789760770.475327:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145600 -> 145608 00010000:00010000:3.0:1789760770.475541:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.475559:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.475567:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.475575:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760770.475579:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.475588:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.475607:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.475613:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef99c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74534912->74539007] (req 74534912->74539007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.475814:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503716224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8451us (8670us total) trans 4294985504 rc 0/0 00000100:00100000:3.0:1789760770.475825:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18208 00000100:00100000:3.0:1789760770.475834:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700580047029 time 700591954004 state 0x2 ref 0 00000001:00080000:3.0:1789760770.502624:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985506 is committed 00000100:00100000:0.0:1789760770.506060:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760770.512130:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985507 is committed 00000100:00100000:3.0:1789760770.522169:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503716736 00000100:00100000:3.0:1789760770.522179:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503716736 00000100:00100000:3.0:1789760770.522232:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700620709157 time 700631036922 state 0x0 ref 1 00000100:00100000:3.0:1789760770.522247:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700631297338 time 700631036922 state 0x1 ref 1 00000100:00100000:3.0:1789760770.522279:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700631297338 time 700646885385 state 0x0 ref 1 00000100:00100000:3.0:1789760770.522291:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503716736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.522335:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74551296, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x238e0 00010000:00010000:3.0:1789760770.522344:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.522359:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.522363:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.522369:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.522381:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.528342:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74551296 -> 74555392 00002000:00010000:3.0:1789760770.528348:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145632 -> 145640 00010000:00010000:3.0:1789760770.528360:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.528510:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.528618:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.528626:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760770.528631:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.528641:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.528669:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.528677:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74551296->74555391] (req 74551296->74555391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.528990:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503716736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6696us (7124us total) trans 4294985508 rc 0/0 00000100:00100000:3.0:1789760770.529000:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18212 00000100:00100000:3.0:1789760770.529010:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700631297338 time 700646885385 state 0x2 ref 0 00000001:00080000:3.0:1789760770.540059:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985509 is committed 00000100:00100000:0.0:1789760770.550228:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760770.558131:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985510 is committed 00000001:00080000:3.0:1789760770.574946:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985511 is committed 00010000:00010000:3.0:1789760770.578713:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760770.599622:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503717248 00000100:00100000:3.0:1789760770.599631:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503717248 00000100:00100000:3.0:1789760770.599918:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700675607867 time 700686155391 state 0x0 ref 1 00000100:00100000:3.0:1789760770.600032:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700686415807 time 700686155391 state 0x1 ref 1 00000100:00100000:3.0:1789760770.600066:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700686415807 time 700724673309 state 0x0 ref 1 00000100:00100000:3.0:1789760770.600079:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503717248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.600125:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74567680, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23900 00010000:00010000:3.0:1789760770.600134:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.600150:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.600154:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.600160:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74567680->74571775] (req 74567680->74571775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.600174:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083ef9d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74567680->74571775] (req 74567680->74571775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.609525:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985512 is committed 00000100:00100000:0.0:1789760770.616060:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760770.637547:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503717504 00000100:00100000:3.0:1789760770.637559:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503717504 00000100:00100000:0.0:1789760770.639252:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700724933725 time 700740886738 state 0x0 ref 1 00000100:00100000:0.0:1789760770.639272:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700741147154 time 700740886738 state 0x1 ref 1 00000100:00100000:0.0:1789760770.639318:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700741147154 time 700763924568 state 0x0 ref 1 00000100:00100000:0.0:1789760770.639333:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503717504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.639446:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74575872, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23910 00010000:00010000:0.0:1789760770.639455:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.639471:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.639476:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.639483:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.639496:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.652433:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985514 is committed 00002000:00010000:0.0:1789760770.652536:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74575872 -> 74579968 00002000:00010000:0.0:1789760770.652542:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145680 -> 145688 00010000:00010000:0.0:1789760770.652555:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.652568:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.652578:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.652592:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.652597:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.652606:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.652627:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.652635:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74575872->74579967] (req 74575872->74579967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.652738:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503717504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13406us (16380us total) trans 4294985514 rc 0/0 00000100:00100000:0.0:1789760770.652749:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18218 00000100:00100000:0.0:1789760770.652767:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700741147154 time 700763924568 state 0x2 ref 0 00000001:00080000:3.0:1789760770.670011:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985515 is committed 00000001:00080000:3.0:1789760770.698191:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985516 is committed 00000001:00080000:3.0:1789760770.715825:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985517 is committed 00000100:00100000:3.0:1789760770.723839:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503717888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 20665us (20894us total) trans 4294985517 rc 0/0 00000100:00100000:3.0:1789760770.723855:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18221 00000100:00100000:3.0:1789760770.723864:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700804837706 time 700827756832 state 0x2 ref 0 00000100:00100000:0.0:1789760770.732260:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503718016 00000100:00100000:0.0:1789760770.732276:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503718016 00000100:00100000:0.0:1789760770.732339:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700804837706 time 700827756832 state 0x0 ref 1 00000100:00100000:0.0:1789760770.732356:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700828017248 time 700827756832 state 0x1 ref 1 00000100:00100000:0.0:1789760770.733202:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700828017248 time 700857803777 state 0x0 ref 1 00000100:00100000:0.0:1789760770.733225:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503718016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.733275:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74592256, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x23930 00010000:00010000:0.0:1789760770.733284:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.733299:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.733303:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.733309:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.733324:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.743895:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985518 is committed 00002000:00010000:0.0:1789760770.744326:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74592256 -> 74596352 00002000:00010000:0.0:1789760770.744331:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145712 -> 145720 00010000:00010000:0.0:1789760770.744344:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.744356:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.744365:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.744372:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.744377:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.744385:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.744432:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.744439:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083ef9fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74592256->74596351] (req 74592256->74596351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.744520:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503718016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11297us (15636us total) trans 4294985518 rc 0/0 00000100:00100000:0.0:1789760770.744530:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18222 00000100:00100000:0.0:1789760770.744539:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700828017248 time 700857803777 state 0x2 ref 0 00000001:00080000:3.0:1789760770.771798:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985519 is committed 00000001:00080000:3.0:1789760770.796897:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985520 is committed 00000100:00100000:3.0:1789760770.805087:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503718400 00000100:00100000:3.0:1789760770.805097:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503718400 00000100:00100000:3.0:1789760770.805151:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700882224066 time 700913087358 state 0x0 ref 1 00000100:00100000:3.0:1789760770.805165:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700913347774 time 700913087358 state 0x1 ref 1 00000100:00100000:3.0:1789760770.805192:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700913347774 time 700929799154 state 0x0 ref 1 00000100:00100000:3.0:1789760770.805203:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f70700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503718400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.805251:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74604544, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23948 00010000:00010000:3.0:1789760770.805261:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.805276:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.805282:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.805288:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.805306:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.812498:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74604544 -> 74608640 00002000:00010000:3.0:1789760770.812503:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145736 -> 145744 00010000:00010000:3.0:1789760770.812515:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.812526:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.812535:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.812542:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760770.812548:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.812557:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.812577:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.812585:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74604544->74608639] (req 74604544->74608639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.812765:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f70700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503718400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7557us (7825us total) trans 4294985521 rc 0/0 00000100:00100000:3.0:1789760770.812775:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18225 00000100:00100000:3.0:1789760770.812784:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700913347774 time 700929799154 state 0x2 ref 0 00000100:00100000:0.0:1789760770.818585:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503718528 00000100:00100000:0.0:1789760770.818596:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503718528 00000100:00100000:0.0:1789760770.818664:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700913347774 time 700929799154 state 0x0 ref 1 00000100:00100000:0.0:1789760770.818679:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700930059570 time 700929799154 state 0x1 ref 1 00000100:00100000:0.0:1789760770.818812:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700930059570 time 700943417076 state 0x0 ref 1 00000100:00100000:0.0:1789760770.818827:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f70a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503718528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.818873:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74608640, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23950 00010000:00010000:0.0:1789760770.818884:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.818899:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760770.818903:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.818910:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.818924:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.828134:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74608640 -> 74612736 00002000:00010000:0.0:1789760770.828139:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145744 -> 145752 00010000:00010000:0.0:1789760770.828154:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.828165:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.828174:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.828181:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.828186:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.828195:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.828214:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.828222:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74608640->74612735] (req 74608640->74612735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.828321:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f70a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503718528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9492us (10419us total) trans 4294985522 rc 0/0 00000100:00100000:0.0:1789760770.828330:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18226 00000100:00100000:0.0:1789760770.828340:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700930059570 time 700943417076 state 0x2 ref 0 00000100:00100000:0.0:1789760770.870778:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760770.870866:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503718912 00000100:00100000:3.0:1789760770.870876:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503718912 00000100:00100000:3.0:1789760770.870931:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700962920885 time 700982089456 state 0x0 ref 1 00000100:00100000:3.0:1789760770.870954:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700982349872 time 700982089456 state 0x1 ref 1 00000100:00100000:3.0:1789760770.870984:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700982349872 time 700995590934 state 0x0 ref 1 00000100:00100000:3.0:1789760770.870995:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503718912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760770.871045:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74620928, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23968 00010000:00010000:3.0:1789760770.871054:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.871076:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760770.871081:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.871087:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74620928->74625023] (req 74620928->74625023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.871100:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083efa2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74620928->74625023] (req 74620928->74625023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.886250:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985525 is committed 00000100:00100000:0.0:1789760770.898009:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503719040 00000100:00100000:0.0:1789760770.898021:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503719040 00000100:00100000:0.0:1789760770.898092:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700982349872 time 700995590934 state 0x0 ref 1 00000100:00100000:0.0:1789760770.898109:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700995851350 time 700995590934 state 0x1 ref 1 00000100:00100000:0.0:1789760770.898140:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700995851350 time 701022746351 state 0x0 ref 1 00000100:00100000:0.0:1789760770.898152:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503719040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.898202:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74625024, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23970 00010000:00010000:0.0:1789760770.898210:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898231:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760770.898236:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.898242:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898256:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.905467:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74625024 -> 74629120 00002000:00010000:0.0:1789760770.905473:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145776 -> 145784 00010000:00010000:0.0:1789760770.905488:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.905501:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.905511:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.905519:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.905524:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.905534:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.905560:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.905568:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efa36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74625024->74629119] (req 74625024->74629119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.905675:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503719040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7518us (7774us total) trans 4294985526 rc 0/0 00000100:00100000:0.0:1789760770.906487:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18230 00000100:00100000:0.0:1789760770.906503:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 700995851350 time 701022746351 state 0x2 ref 0 00000001:00080000:0.0:1789760770.925347:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985527 is committed 00000100:00100000:0.0:1789760770.933941:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760770.943974:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74633216 -> 74637312 00002000:00010000:0.0:1789760770.943979:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145792 -> 145800 00010000:00010000:0.0:1789760770.943991:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efa44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74633216->74637311] (req 74633216->74637311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.944004:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efa44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74633216->74637311] (req 74633216->74637311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.944278:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efa44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74633216->74637311] (req 74633216->74637311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.944287:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760770.944293:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efa44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74633216->74637311] (req 74633216->74637311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.944302:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efa44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74633216->74637311] (req 74633216->74637311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.944322:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efa44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74633216->74637311] (req 74633216->74637311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.944330:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efa44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74633216->74637311] (req 74633216->74637311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.944628:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503719296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8054us (10691us total) trans 4294985528 rc 0/0 00000100:00100000:0.0:1789760770.944640:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18232 00000100:00100000:0.0:1789760770.944661:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701041641337 time 701061163654 state 0x2 ref 0 00000100:00100000:3.0:1789760770.951132:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760770.957054:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985529 is committed 00000100:00100000:3.0:1789760770.960993:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760770.961272:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503719552 00000100:00100000:0.0:1789760770.961283:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503719552 00000100:00100000:0.0:1789760770.961342:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701061424070 time 701076215758 state 0x0 ref 1 00000100:00100000:0.0:1789760770.961357:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701076476174 time 701076215758 state 0x1 ref 1 00000100:00100000:0.0:1789760770.961433:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701076476174 time 701086039510 state 0x0 ref 1 00000100:00100000:0.0:1789760770.961446:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503719552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760770.961494:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74641408, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23990 00010000:00010000:0.0:1789760770.961503:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efa52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.961518:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760770.961523:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.961529:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efa52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74641408->74645503] (req 74641408->74645503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.961542:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efa52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74641408->74645503] (req 74641408->74645503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.972066:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985530 is committed 00000100:00100000:0.0:1789760770.975903:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760770.981430:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985531 is committed 00000100:00100000:3.0:1789760770.987359:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760770.992947:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985532 is committed 00000001:00080000:0.0:1789760771.004787:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985533 is committed 00000100:00100000:0.0:1789760771.014094:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760771.024791:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74657792 -> 74661888 00002000:00010000:0.0:1789760771.024797:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145840 -> 145848 00010000:00010000:0.0:1789760771.024809:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efa6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74657792->74661887] (req 74657792->74661887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.024821:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efa6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74657792->74661887] (req 74657792->74661887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.024829:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efa6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74657792->74661887] (req 74657792->74661887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.024841:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.024846:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efa6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74657792->74661887] (req 74657792->74661887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.024854:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efa6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74657792->74661887] (req 74657792->74661887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.024874:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efa6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74657792->74661887] (req 74657792->74661887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.024882:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efa6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74657792->74661887] (req 74657792->74661887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.025108:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503720064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10681us (11019us total) trans 4294985534 rc 0/0 00000100:00100000:0.0:1789760771.025120:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18238 00000100:00100000:0.0:1789760771.025130:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701123092569 time 701138984331 state 0x2 ref 0 00000100:00100000:3.0:1789760771.029229:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.051590:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985536 is committed 00000100:00100000:0.0:1789760771.056636:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760771.084091:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.088901:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503720576 00000100:00100000:0.0:1789760771.088915:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503720576 00000100:00100000:0.0:1789760771.089863:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701169951445 time 701181479855 state 0x0 ref 1 00000100:00100000:0.0:1789760771.089881:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701181740271 time 701181479855 state 0x1 ref 1 00000100:00100000:0.0:1789760771.089920:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701181740271 time 701214526626 state 0x0 ref 1 00000100:00100000:0.0:1789760771.089936:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503720576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.090121:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74674176, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x239d0 00010000:00010000:0.0:1789760771.090132:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.090147:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.090152:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.090843:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.090866:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.102592:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74674176 -> 74678272 00002000:00010000:0.0:1789760771.102597:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145872 -> 145880 00010000:00010000:0.0:1789760771.102611:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102624:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102634:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102658:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.102664:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102675:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102694:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102702:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efa8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74674176->74678271] (req 74674176->74678271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.102794:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503720576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12859us (19635us total) trans 4294985538 rc 0/0 00000100:00100000:0.0:1789760771.102803:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18242 00000100:00100000:0.0:1789760771.102814:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701181740271 time 701214526626 state 0x2 ref 0 00000100:00100000:3.0:1789760771.143539:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.151804:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503720960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8064us (8268us total) trans 4294985541 rc 0/0 00000100:00100000:0.0:1789760771.151816:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18245 00000100:00100000:0.0:1789760771.151826:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701255183264 time 701268329718 state 0x2 ref 0 00000100:00100000:0.0:1789760771.155524:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503721088 00000100:00100000:0.0:1789760771.155536:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503721088 00000100:00100000:0.0:1789760771.155600:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701255183264 time 701268329718 state 0x0 ref 1 00000100:00100000:0.0:1789760771.155616:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701268590134 time 701268329718 state 0x1 ref 1 00000100:00100000:0.0:1789760771.155639:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701268590134 time 701280245868 state 0x0 ref 1 00000100:00100000:0.0:1789760771.155663:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503721088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.155707:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74690560, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x239f0 00010000:00010000:0.0:1789760771.155716:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.155731:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.155736:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.155742:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.155756:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.161085:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74690560 -> 74694656 00002000:00010000:0.0:1789760771.161090:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145904 -> 145912 00010000:00010000:0.0:1789760771.161190:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161205:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161219:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161226:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.161231:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161240:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161259:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.161267:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaa6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74690560->74694655] (req 74690560->74694655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.161525:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503721088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5870us (6650us total) trans 4294985542 rc 0/0 00000100:00100000:0.0:1789760771.161536:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18246 00000100:00100000:0.0:1789760771.161545:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701268590134 time 701280245868 state 0x2 ref 0 00000100:00100000:3.0:1789760771.166145:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760771.176519:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760771.185542:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74698752 -> 74702848 00002000:00010000:0.0:1789760771.185548:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145920 -> 145928 00010000:00010000:0.0:1789760771.185562:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efab4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74698752->74702847] (req 74698752->74702847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.185579:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efab4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74698752->74702847] (req 74698752->74702847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.185588:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efab4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74698752->74702847] (req 74698752->74702847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.185595:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.185600:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efab4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74698752->74702847] (req 74698752->74702847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.185610:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efab4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74698752->74702847] (req 74698752->74702847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.185630:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efab4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74698752->74702847] (req 74698752->74702847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.185638:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efab4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74698752->74702847] (req 74698752->74702847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.185754:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503721344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8967us (9312us total) trans 4294985544 rc 0/0 00000100:00100000:0.0:1789760771.185765:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18248 00000100:00100000:0.0:1789760771.185774:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701291204694 time 701301382334 state 0x2 ref 0 00000100:00100000:0.0:1789760771.189516:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503721472 00000100:00100000:0.0:1789760771.189526:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503721472 00000100:00100000:0.0:1789760771.189582:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701291204694 time 701301382334 state 0x0 ref 1 00000100:00100000:0.0:1789760771.189597:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701301642750 time 701301382334 state 0x1 ref 1 00000100:00100000:0.0:1789760771.189628:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701301642750 time 701314234195 state 0x0 ref 1 00000100:00100000:0.0:1789760771.189640:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffb6d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503721472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.189709:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74702848, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a08 00010000:00010000:0.0:1789760771.189719:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.189735:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.189740:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.189747:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.189761:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.197805:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74702848 -> 74706944 00002000:00010000:0.0:1789760771.197811:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145928 -> 145936 00010000:00010000:0.0:1789760771.197823:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.197836:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.197850:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.197857:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.197862:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.197871:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.197891:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.197899:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efabb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74702848->74706943] (req 74702848->74706943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.198092:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffb6d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503721472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8449us (8770us total) trans 4294985545 rc 0/0 00000100:00100000:0.0:1789760771.198107:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18249 00000100:00100000:0.0:1789760771.198116:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701301642750 time 701314234195 state 0x2 ref 0 00000100:00100000:0.0:1789760771.201352:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503721600 00000100:00100000:0.0:1789760771.201362:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503721600 00000100:00100000:0.0:1789760771.201552:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701301642750 time 701314234195 state 0x0 ref 1 00000100:00100000:0.0:1789760771.201569:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701314494611 time 701314234195 state 0x1 ref 1 00000100:00100000:0.0:1789760771.201660:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701314494611 time 701326253134 state 0x0 ref 1 00000100:00100000:0.0:1789760771.201675:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503721600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.201724:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74706944, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a10 00010000:00010000:0.0:1789760771.201732:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efac2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.201747:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.201751:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.201757:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74706944->74711039] (req 74706944->74711039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.201770:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74706944->74711039] (req 74706944->74711039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.216327:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760771.231547:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.231875:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503721856 00000100:00100000:0.0:1789760771.231885:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503721856 00000100:00100000:0.0:1789760771.231939:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701326513550 time 701342117464 state 0x0 ref 1 00000100:00100000:0.0:1789760771.231966:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701342377880 time 701342117464 state 0x1 ref 1 00000100:00100000:0.0:1789760771.231998:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701342377880 time 701356604958 state 0x0 ref 1 00000100:00100000:0.0:1789760771.232019:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503721856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.232069:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74715136, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a20 00010000:00010000:0.0:1789760771.232079:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.232095:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.232312:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.232321:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.232336:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.240454:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74715136 -> 74719232 00002000:00010000:0.0:1789760771.240460:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145952 -> 145960 00010000:00010000:0.0:1789760771.240474:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.240489:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.240498:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.240506:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.240511:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.240521:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.240543:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.240551:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74715136->74719231] (req 74715136->74719231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.240658:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503721856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8635us (9106us total) trans 4294985548 rc 0/0 00000100:00100000:0.0:1789760771.240677:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18252 00000100:00100000:0.0:1789760771.240687:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701342377880 time 701356604958 state 0x2 ref 0 00000100:00100000:0.0:1789760771.247510:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503721984 00000100:00100000:0.0:1789760771.247519:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503721984 00000100:00100000:0.0:1789760771.247571:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701342377880 time 701356604958 state 0x0 ref 1 00000100:00100000:0.0:1789760771.247586:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701356865374 time 701356604958 state 0x1 ref 1 00000100:00100000:0.0:1789760771.247616:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701356865374 time 701372222551 state 0x0 ref 1 00000100:00100000:0.0:1789760771.247627:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503721984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.247687:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74719232, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a28 00010000:00010000:0.0:1789760771.247696:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.247710:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.247715:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.247721:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.247735:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.254636:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74719232 -> 74723328 00002000:00010000:0.0:1789760771.254642:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145960 -> 145968 00010000:00010000:0.0:1789760771.254665:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.254679:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.254698:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.254706:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.254711:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.254718:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.254910:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.254923:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efad7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74719232->74723327] (req 74719232->74723327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.255016:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503721984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7387us (7589us total) trans 4294985549 rc 0/0 00000100:00100000:0.0:1789760771.255026:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18253 00000100:00100000:0.0:1789760771.255036:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701356865374 time 701372222551 state 0x2 ref 0 00000100:00100000:3.0:1789760771.259617:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.275740:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503722240 00000100:00100000:0.0:1789760771.275754:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503722240 00010000:00010000:1.0:1789760771.281013:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efae5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74727424->74731519] (req 74727424->74731519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.281107:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503722240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5055us (6919us total) trans 4294985551 rc 0/0 00000100:00100000:1.0:1789760771.281118:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18255 00000100:00100000:1.0:1789760771.281127:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701384685374 time 701400641570 state 0x2 ref 0 00000100:00100000:1.0:1789760771.283726:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523605888:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760771.283803:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760771.284015:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523605888 00000100:00100000:0.0:1789760771.284023:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523605888 00000100:00100000:0.0:1789760771.284073:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1789760771.284082:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523605888:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760771.284112:0:7941:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:0.0:1789760771.284170:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523605888:12345-0@lo:41:osp-pre-1-0.0 Request processed in 89us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760771.284177:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:3.0:1789760771.284251:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523605888:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760771.287020:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.287173:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503722368 00000100:00100000:0.0:1789760771.287182:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503722368 00000100:00100000:0.0:1789760771.287233:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701384685374 time 701400641570 state 0x0 ref 1 00000100:00100000:0.0:1789760771.287248:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701400901986 time 701400641570 state 0x1 ref 1 00000100:00100000:0.0:1789760771.287276:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701400901986 time 701411882918 state 0x0 ref 1 00000100:00100000:0.0:1789760771.287290:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503722368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.287333:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74731520, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a40 00010000:00010000:0.0:1789760771.287341:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.287356:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.287360:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.287366:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.287386:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.295083:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74731520 -> 74735616 00002000:00010000:0.0:1789760771.295089:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145984 -> 145992 00010000:00010000:0.0:1789760771.295102:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.295115:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.295124:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.295131:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.295136:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.295145:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.295164:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.295171:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efaec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74731520->74735615] (req 74731520->74735615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.295307:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503722368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8013us (8291us total) trans 4294985552 rc 0/0 00000100:00100000:0.0:1789760771.295318:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18256 00000100:00100000:0.0:1789760771.295328:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701400901986 time 701411882918 state 0x2 ref 0 00000100:00100000:3.0:1789760771.299506:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.299608:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503722496 00000100:00100000:0.0:1789760771.299618:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503722496 00000100:00100000:0.0:1789760771.299679:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701400901986 time 701411882918 state 0x0 ref 1 00000100:00100000:0.0:1789760771.299703:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701412143334 time 701411882918 state 0x1 ref 1 00000100:00100000:0.0:1789760771.299727:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701412143334 time 701424334616 state 0x0 ref 1 00000100:00100000:0.0:1789760771.299814:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503722496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.299862:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74735616, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a48 00010000:00010000:0.0:1789760771.299872:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.299887:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.299892:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.299898:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.299911:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.305008:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74735616 -> 74739712 00002000:00010000:0.0:1789760771.305014:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 145992 -> 146000 00010000:00010000:0.0:1789760771.305027:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.305040:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.305049:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.305058:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.305062:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.305071:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.305090:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.305098:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efaf3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74735616->74739711] (req 74735616->74739711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.305237:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503722496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5424us (5735us total) trans 4294985553 rc 0/0 00000100:00100000:0.0:1789760771.305248:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18257 00000100:00100000:0.0:1789760771.305256:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701412143334 time 701424334616 state 0x2 ref 0 00000100:00100000:1.0:1789760771.309891:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760771.317103:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985554 is committed 00002000:00010000:3.0:1789760771.317235:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74739712 -> 74743808 00002000:00010000:3.0:1789760771.317241:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146000 -> 146008 00010000:00010000:3.0:1789760771.317255:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efafa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74739712->74743807] (req 74739712->74743807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.317268:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efafa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74739712->74743807] (req 74739712->74743807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.317277:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efafa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74739712->74743807] (req 74739712->74743807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.317290:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760771.317295:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efafa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74739712->74743807] (req 74739712->74743807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.317303:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efafa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74739712->74743807] (req 74739712->74743807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.317322:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efafa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74739712->74743807] (req 74739712->74743807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.317330:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083efafa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74739712->74743807] (req 74739712->74743807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.317567:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503722624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7274us (7680us total) trans 4294985554 rc 0/0 00000100:00100000:3.0:1789760771.317579:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18258 00000100:00100000:3.0:1789760771.317588:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701424595032 time 701434884285 state 0x2 ref 0 00000100:00100000:1.0:1789760771.320252:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760771.325449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985555 is committed 00000100:00100000:1.0:1789760771.330778:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.330920:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503722880 00000100:00100000:0.0:1789760771.330935:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503722880 00000100:00100000:0.0:1789760771.330992:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701435144701 time 701445094821 state 0x0 ref 1 00000100:00100000:0.0:1789760771.331008:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701445355237 time 701445094821 state 0x1 ref 1 00000100:00100000:0.0:1789760771.331037:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701445355237 time 701455644158 state 0x0 ref 1 00000100:00100000:0.0:1789760771.331049:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503722880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.331097:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74747904, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a60 00010000:00010000:0.0:1789760771.331106:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.331121:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.331126:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.331133:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.331145:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.337786:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74747904 -> 74752000 00002000:00010000:0.0:1789760771.337791:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146016 -> 146024 00010000:00010000:0.0:1789760771.337807:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.337903:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.337912:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.337920:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.337924:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.337934:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.337954:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.337966:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74747904->74751999] (req 74747904->74751999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.338056:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503722880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7003us (7282us total) trans 4294985556 rc 0/0 00000100:00100000:0.0:1789760771.338067:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18260 00000100:00100000:0.0:1789760771.338076:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701445355237 time 701455644158 state 0x2 ref 0 00000100:00100000:1.0:1789760771.341094:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.341214:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503723008 00000100:00100000:0.0:1789760771.341225:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503723008 00000100:00100000:0.0:1789760771.341280:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701445355237 time 701455644158 state 0x0 ref 1 00000100:00100000:0.0:1789760771.341294:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701455904574 time 701455644158 state 0x1 ref 1 00000100:00100000:0.0:1789760771.341323:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701455904574 time 701465930058 state 0x0 ref 1 00000100:00100000:0.0:1789760771.341336:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503723008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.341424:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74752000, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a68 00010000:00010000:0.0:1789760771.341434:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.341448:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.341453:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.341459:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.341477:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.347911:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74752000 -> 74756096 00002000:00010000:1.0:1789760771.347916:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146024 -> 146032 00010000:00010000:1.0:1789760771.347930:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.347943:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.347953:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.347966:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.347971:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.347979:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.347998:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.348005:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efb0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74752000->74756095] (req 74752000->74756095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.348141:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503723008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6803us (7052us total) trans 4294985557 rc 0/0 00000100:00100000:1.0:1789760771.348151:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18261 00000100:00100000:1.0:1789760771.348161:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701455904574 time 701465930058 state 0x2 ref 0 00000100:00100000:1.0:1789760771.351752:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760771.352266:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503723136 00000100:00100000:3.0:1789760771.352278:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503723136 00000100:00100000:3.0:1789760771.352334:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701455904574 time 701465930058 state 0x0 ref 1 00000100:00100000:3.0:1789760771.352349:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701466190474 time 701465930058 state 0x1 ref 1 00000100:00100000:3.0:1789760771.352421:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701466190474 time 701477026925 state 0x0 ref 1 00000100:00100000:3.0:1789760771.352437:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503723136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760771.352483:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74756096, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a70 00010000:00010000:3.0:1789760771.352492:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.352506:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760771.352511:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.352517:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.352531:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.364929:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74756096 -> 74760192 00002000:00010000:3.0:1789760771.364940:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146032 -> 146040 00010000:00010000:3.0:1789760771.364954:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.364966:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.364974:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.364981:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760771.364986:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.364995:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.365014:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.365021:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efb16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74756096->74760191] (req 74756096->74760191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.366588:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503723136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14140us (14832us total) trans 4294985558 rc 0/0 00000100:00100000:3.0:1789760771.366603:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18262 00000100:00100000:3.0:1789760771.366614:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701466190474 time 701477026925 state 0x2 ref 0 00000100:00100000:3.0:1789760771.373251:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.381542:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985559 is committed 00000100:00100000:1.0:1789760771.388106:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.390953:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503723392 00000100:00100000:0.0:1789760771.390971:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503723392 00000100:00100000:0.0:1789760771.391035:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701477287341 time 701498114929 state 0x0 ref 1 00000100:00100000:0.0:1789760771.391050:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701498375345 time 701498114929 state 0x1 ref 1 00000100:00100000:0.0:1789760771.391135:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701498375345 time 701515741066 state 0x0 ref 1 00000100:00100000:0.0:1789760771.391149:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503723392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.391193:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74764288, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23a80 00010000:00010000:0.0:1789760771.391202:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.391216:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.391221:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.391227:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.391238:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.399575:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74764288 -> 74768384 00002000:00010000:0.0:1789760771.399584:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146048 -> 146056 00010000:00010000:0.0:1789760771.399599:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.399613:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.399621:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.399629:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.399634:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.399642:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.399672:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.399680:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74764288->74768383] (req 74764288->74768383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.399911:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503723392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8760us (11809us total) trans 4294985560 rc 0/0 00000100:00100000:0.0:1789760771.399923:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18264 00000100:00100000:0.0:1789760771.399932:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701498375345 time 701515741066 state 0x2 ref 0 00000100:00100000:1.0:1789760771.409883:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.414496:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985561 is committed 00000100:00100000:3.0:1789760771.419612:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.423904:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985562 is committed 00000100:00100000:1.0:1789760771.426728:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.431201:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985563 is committed 00000100:00100000:1.0:1789760771.437172:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.437469:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503723904 00000100:00100000:0.0:1789760771.437480:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503723904 00000100:00100000:0.0:1789760771.437536:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701544770567 time 701551495076 state 0x0 ref 1 00000100:00100000:0.0:1789760771.437551:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701551755492 time 701551495076 state 0x1 ref 1 00000100:00100000:0.0:1789760771.437584:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701551755492 time 701562190757 state 0x0 ref 1 00000100:00100000:0.0:1789760771.437597:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503723904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.437642:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74780672, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23aa0 00010000:00010000:0.0:1789760771.437662:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.437679:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.437683:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.437689:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.437702:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.443012:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74780672 -> 74784768 00002000:00010000:0.0:1789760771.443017:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146080 -> 146088 00010000:00010000:0.0:1789760771.443029:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.443042:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.443051:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.443059:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.443063:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.443085:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.443104:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.443113:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083efb40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74780672->74784767] (req 74780672->74784767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.443205:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503723904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5605us (6039us total) trans 4294985564 rc 0/0 00000100:00100000:0.0:1789760771.443216:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18268 00000100:00100000:0.0:1789760771.443227:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701551755492 time 701562190757 state 0x2 ref 0 00000100:00100000:3.0:1789760771.447463:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760771.447558:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503724032 00000100:00100000:1.0:1789760771.447568:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503724032 00000100:00100000:1.0:1789760771.447620:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701551755492 time 701562190757 state 0x0 ref 1 00000100:00100000:1.0:1789760771.447635:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701562451173 time 701562190757 state 0x1 ref 1 00000100:00100000:1.0:1789760771.447679:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701562451173 time 701572285602 state 0x0 ref 1 00000100:00100000:1.0:1789760771.447699:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503724032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.447745:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74784768, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23aa8 00010000:00010000:1.0:1789760771.447755:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447770:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760771.447774:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.447780:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447793:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.459690:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985565 is committed 00002000:00010000:1.0:1789760771.460021:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74784768 -> 74788864 00002000:00010000:1.0:1789760771.460026:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146088 -> 146096 00010000:00010000:1.0:1789760771.460040:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.460053:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.460068:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.460076:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.460080:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.460089:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.460107:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.460115:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efb47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74784768->74788863] (req 74784768->74788863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.460205:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503724032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12503us (12746us total) trans 4294985565 rc 0/0 00000100:00100000:1.0:1789760771.460215:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18269 00000100:00100000:1.0:1789760771.460225:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701562451173 time 701572285602 state 0x2 ref 0 00000100:00100000:1.0:1789760771.464514:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760771.473485:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985566 is committed 00000100:00100000:0.0:1789760771.480175:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760771.491009:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985567 is committed 00000100:00100000:3.0:1789760771.495185:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.495346:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503724416 00000100:00100000:0.0:1789760771.495356:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503724416 00000100:00100000:0.0:1789760771.495458:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701589996185 time 701604991231 state 0x0 ref 1 00000100:00100000:0.0:1789760771.495474:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701605251647 time 701604991231 state 0x1 ref 1 00000100:00100000:0.0:1789760771.495499:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701605251647 time 701620105887 state 0x0 ref 1 00000100:00100000:0.0:1789760771.495511:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503724416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.495560:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74797056, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23ac0 00010000:00010000:0.0:1789760771.495569:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.495584:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.495588:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.495605:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.495619:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.505022:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985568 is committed 00002000:00010000:0.0:1789760771.505137:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74797056 -> 74801152 00002000:00010000:0.0:1789760771.505143:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146112 -> 146120 00010000:00010000:0.0:1789760771.505155:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.505168:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.505177:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.505185:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.505189:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.505199:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.505217:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.505224:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74797056->74801151] (req 74797056->74801151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.505316:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503724416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9802us (10137us total) trans 4294985568 rc 0/0 00000100:00100000:0.0:1789760771.505326:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18272 00000100:00100000:0.0:1789760771.505336:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701605251647 time 701620105887 state 0x2 ref 0 00000100:00100000:1.0:1789760771.507861:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:1.0:1789760771.519876:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74801152 -> 74805248 00002000:00010000:1.0:1789760771.519883:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146120 -> 146128 00010000:00010000:1.0:1789760771.519945:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efb63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74801152->74805247] (req 74801152->74805247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.519960:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efb63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74801152->74805247] (req 74801152->74805247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.519969:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efb63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74801152->74805247] (req 74801152->74805247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.519977:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.519982:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efb63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74801152->74805247] (req 74801152->74805247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.519991:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efb63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74801152->74805247] (req 74801152->74805247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.520010:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efb63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74801152->74805247] (req 74801152->74805247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.520018:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efb63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74801152->74805247] (req 74801152->74805247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.520485:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503724544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11679us (12513us total) trans 4294985569 rc 0/0 00000100:00100000:1.0:1789760771.520500:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18273 00000100:00100000:1.0:1789760771.520511:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701620366303 time 701633281962 state 0x2 ref 0 00000100:00100000:1.0:1789760771.525224:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760771.535756:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985570 is committed 00000100:00100000:3.0:1789760771.543147:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760771.543492:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503724800 00000100:00100000:1.0:1789760771.543502:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503724800 00000100:00100000:1.0:1789760771.543564:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701633542378 time 701650040837 state 0x0 ref 1 00000100:00100000:1.0:1789760771.543580:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701650301253 time 701650040837 state 0x1 ref 1 00000100:00100000:1.0:1789760771.543675:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701650301253 time 701668280724 state 0x0 ref 1 00000100:00100000:1.0:1789760771.544347:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503724800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.544564:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74809344, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23ad8 00010000:00010000:1.0:1789760771.544572:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.544585:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760771.544588:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.544593:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.544602:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.552665:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985571 is committed 00002000:00010000:0.0:1789760771.554243:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74809344 -> 74813440 00002000:00010000:0.0:1789760771.554253:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146136 -> 146144 00010000:00010000:0.0:1789760771.554268:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.555205:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.555233:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.555241:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.555247:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.555263:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.555292:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.555933:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efb71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74809344->74813439] (req 74809344->74813439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.556161:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503724800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11808us (13009us total) trans 4294985571 rc 0/0 00000100:00100000:0.0:1789760771.556172:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18275 00000100:00100000:0.0:1789760771.556182:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701650301253 time 701668280724 state 0x2 ref 0 00000100:00100000:1.0:1789760771.560987:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.561830:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503724928 00000100:00100000:0.0:1789760771.561841:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503724928 00000100:00100000:0.0:1789760771.561896:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701650301253 time 701668280724 state 0x0 ref 1 00000100:00100000:0.0:1789760771.561913:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701668541140 time 701668280724 state 0x1 ref 1 00000100:00100000:0.0:1789760771.562046:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701668541140 time 701686650858 state 0x0 ref 1 00000100:00100000:0.0:1789760771.562060:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503724928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.562109:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74813440, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23ae0 00010000:00010000:0.0:1789760771.562118:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.562135:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.562139:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.562146:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.562172:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.571668:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985572 is committed 00002000:00010000:0.0:1789760771.571952:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74813440 -> 74817536 00002000:00010000:0.0:1789760771.571958:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146144 -> 146152 00010000:00010000:0.0:1789760771.571971:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.571984:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.571992:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.571999:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.572003:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.572013:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.572033:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.572041:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74813440->74817535] (req 74813440->74817535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.572250:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503724928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10188us (11270us total) trans 4294985572 rc 0/0 00000100:00100000:0.0:1789760771.572261:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18276 00000100:00100000:0.0:1789760771.572270:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701668541140 time 701686650858 state 0x2 ref 0 00000100:00100000:1.0:1789760771.574866:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:1.0:1789760771.586491:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74817536 -> 74821632 00002000:00010000:1.0:1789760771.586497:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146152 -> 146160 00010000:00010000:1.0:1789760771.586511:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083efb7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74817536->74821631] (req 74817536->74821631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.586524:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083efb7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74817536->74821631] (req 74817536->74821631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.586533:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083efb7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74817536->74821631] (req 74817536->74821631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.586541:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.586545:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083efb7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74817536->74821631] (req 74817536->74821631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.586554:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083efb7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74817536->74821631] (req 74817536->74821631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.586573:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083efb7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74817536->74821631] (req 74817536->74821631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.586580:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083efb7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74817536->74821631] (req 74817536->74821631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.586776:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11160us (11918us total) trans 4294985573 rc 0/0 00000100:00100000:1.0:1789760771.586786:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18277 00000100:00100000:1.0:1789760771.586796:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701686911274 time 701700203710 state 0x2 ref 0 00000100:00100000:3.0:1789760771.591383:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00010000:00010000:3.0:1789760771.602522:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760771.605149:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74821632 -> 74825728 00002000:00010000:3.0:1789760771.605162:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146160 -> 146168 00010000:00010000:3.0:1789760771.605177:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efb86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74821632->74825727] (req 74821632->74825727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.605191:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efb86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74821632->74825727] (req 74821632->74825727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.605201:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efb86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74821632->74825727] (req 74821632->74825727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.605209:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760771.605214:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efb86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74821632->74825727] (req 74821632->74825727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.605224:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efb86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74821632->74825727] (req 74821632->74825727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.605243:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efb86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74821632->74825727] (req 74821632->74825727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.605251:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efb86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74821632->74825727] (req 74821632->74825727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.605357:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503725184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13546us (14054us total) trans 4294985574 rc 0/0 00000100:00100000:3.0:1789760771.605377:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18278 00000100:00100000:3.0:1789760771.605418:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701700464126 time 701716287014 state 0x2 ref 0 00000100:00100000:3.0:1789760771.612348:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.612859:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503725312 00000100:00100000:0.0:1789760771.612870:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503725312 00000100:00100000:0.0:1789760771.612932:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701700464126 time 701716287014 state 0x0 ref 1 00000100:00100000:0.0:1789760771.612948:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701716547430 time 701716287014 state 0x1 ref 1 00000100:00100000:0.0:1789760771.612973:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701716547430 time 701737580126 state 0x0 ref 1 00000100:00100000:0.0:1789760771.612986:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.613033:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74825728, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23af8 00010000:00010000:0.0:1789760771.613042:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.613058:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.613063:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.613069:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.613083:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.622229:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985575 is committed 00002000:00010000:1.0:1789760771.622471:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74825728 -> 74829824 00002000:00010000:1.0:1789760771.622477:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146168 -> 146176 00010000:00010000:1.0:1789760771.622490:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.622502:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.622512:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.622519:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.622524:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.622534:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.622566:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.622803:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efb8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74825728->74829823] (req 74825728->74829823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.622924:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9927us (10572us total) trans 4294985575 rc 0/0 00000100:00100000:1.0:1789760771.622934:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18279 00000100:00100000:1.0:1789760771.622951:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701716547430 time 701737580126 state 0x2 ref 0 00000100:00100000:1.0:1789760771.627788:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.630878:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503725440 00000100:00100000:0.0:1789760771.630894:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503725440 00000100:00100000:0.0:1789760771.631022:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701716547430 time 701737580126 state 0x0 ref 1 00000100:00100000:0.0:1789760771.631037:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701737840542 time 701737580126 state 0x1 ref 1 00000100:00100000:0.0:1789760771.631145:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701737840542 time 701755749914 state 0x0 ref 1 00000100:00100000:0.0:1789760771.631170:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.631227:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74829824, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23b00 00010000:00010000:0.0:1789760771.631242:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.631261:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.631265:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.631273:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.631298:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.637635:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985576 is committed 00002000:00010000:0.0:1789760771.637771:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74829824 -> 74833920 00002000:00010000:0.0:1789760771.637776:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146176 -> 146184 00010000:00010000:0.0:1789760771.637790:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.637803:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.637817:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.637825:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.637830:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.637839:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.637858:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.637866:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efb94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74829824->74833919] (req 74829824->74833919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.637968:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6794us (10181us total) trans 4294985576 rc 0/0 00000100:00100000:0.0:1789760771.637978:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18280 00000100:00100000:0.0:1789760771.637989:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701737840542 time 701755749914 state 0x2 ref 0 00000100:00100000:3.0:1789760771.640948:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760771.641672:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503725568 00000100:00100000:1.0:1789760771.641683:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503725568 00000100:00100000:1.0:1789760771.641742:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701737840542 time 701755749914 state 0x0 ref 1 00000100:00100000:1.0:1789760771.641758:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701756010330 time 701755749914 state 0x1 ref 1 00000100:00100000:1.0:1789760771.641822:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701756010330 time 701766428078 state 0x0 ref 1 00000100:00100000:1.0:1789760771.641835:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.641890:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74833920, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23b08 00010000:00010000:1.0:1789760771.641900:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.641915:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760771.641920:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.641927:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.641952:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.651532:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985577 is committed 00002000:00010000:1.0:1789760771.652524:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74833920 -> 74838016 00002000:00010000:1.0:1789760771.652532:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146184 -> 146192 00010000:00010000:1.0:1789760771.652548:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.652561:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.652570:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.652577:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.652581:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.652650:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.652673:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.652682:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efb9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74833920->74838015] (req 74833920->74838015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.652827:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10979us (11881us total) trans 4294985577 rc 0/0 00000100:00100000:1.0:1789760771.652838:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18281 00000100:00100000:1.0:1789760771.652847:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701756010330 time 701766428078 state 0x2 ref 0 00000100:00100000:3.0:1789760771.662915:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.669881:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985578 is committed 00000100:00100000:0.0:1789760771.675610:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760771.675693:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503725824 00000100:00100000:1.0:1789760771.675703:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503725824 00000100:00100000:1.0:1789760771.675758:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701766688494 time 701787984587 state 0x0 ref 1 00000100:00100000:1.0:1789760771.675774:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701788245003 time 701787984587 state 0x1 ref 1 00000100:00100000:1.0:1789760771.675809:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701788245003 time 701800415145 state 0x0 ref 1 00000100:00100000:1.0:1789760771.675823:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.675872:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74842112, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23b18 00010000:00010000:1.0:1789760771.675882:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.675898:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760771.675903:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.675910:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.675926:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.681291:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985579 is committed 00002000:00010000:1.0:1789760771.681453:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74842112 -> 74846208 00002000:00010000:1.0:1789760771.681458:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146200 -> 146208 00010000:00010000:1.0:1789760771.681471:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.681485:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.681495:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.681502:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.681507:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.681516:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.681538:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.681545:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efba9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74842112->74846207] (req 74842112->74846207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.681700:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503725824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5875us (6093us total) trans 4294985579 rc 0/0 00000100:00100000:1.0:1789760771.681711:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18283 00000100:00100000:1.0:1789760771.681720:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701788245003 time 701800415145 state 0x2 ref 0 00000100:00100000:0.0:1789760771.686097:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503725952 00000100:00100000:0.0:1789760771.686107:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503725952 00000100:00100000:0.0:1789760771.686162:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701788245003 time 701800415145 state 0x0 ref 1 00000100:00100000:0.0:1789760771.686178:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701800675561 time 701800415145 state 0x1 ref 1 00000100:00100000:0.0:1789760771.686249:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701800675561 time 701810854621 state 0x0 ref 1 00000100:00100000:0.0:1789760771.686264:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503725952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.686313:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74846208, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23b20 00010000:00010000:0.0:1789760771.686323:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.686339:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.686343:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.686350:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.686365:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.694118:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74846208 -> 74850304 00002000:00010000:0.0:1789760771.694175:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146208 -> 146216 00010000:00010000:0.0:1789760771.694189:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.694202:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.694213:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.694220:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.694225:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.694234:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.694253:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.694262:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74846208->74850303] (req 74846208->74850303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.694356:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503725952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8092us (8453us total) trans 4294985580 rc 0/0 00000100:00100000:0.0:1789760771.694367:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18284 00000100:00100000:0.0:1789760771.694378:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701800675561 time 701810854621 state 0x2 ref 0 00000100:00100000:3.0:1789760771.697660:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.702666:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985581 is committed 00000100:00100000:1.0:1789760771.708087:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.716097:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985582 is committed 00000100:00100000:1.0:1789760771.724939:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.725042:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503726336 00000100:00100000:0.0:1789760771.725052:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503726336 00000100:00100000:0.0:1789760771.725112:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701822740082 time 701833054720 state 0x0 ref 1 00000100:00100000:0.0:1789760771.725127:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701833315136 time 701833054720 state 0x1 ref 1 00000100:00100000:0.0:1789760771.725165:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701833315136 time 701849771361 state 0x0 ref 1 00000100:00100000:0.0:1789760771.725178:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503726336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.725225:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74858496, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23b38 00010000:00010000:0.0:1789760771.725234:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.725249:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.725254:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.725260:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.725290:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.734091:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985583 is committed 00002000:00010000:1.0:1789760771.734292:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74858496 -> 74862592 00002000:00010000:1.0:1789760771.734298:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146232 -> 146240 00010000:00010000:1.0:1789760771.734311:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734322:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734331:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734338:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.734342:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734351:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734379:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734420:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efbc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74858496->74862591] (req 74858496->74862591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.734520:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503726336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9340us (9585us total) trans 4294985583 rc 0/0 00000100:00100000:1.0:1789760771.734530:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18287 00000100:00100000:1.0:1789760771.734540:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701833315136 time 701849771361 state 0x2 ref 0 00000100:00100000:3.0:1789760771.739451:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.747018:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985584 is committed 00000100:00100000:0.0:1789760771.754301:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760771.760241:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985585 is committed 00000100:00100000:1.0:1789760771.766976:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760771.767049:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503726720 00000100:00100000:3.0:1789760771.767060:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503726720 00000100:00100000:3.0:1789760771.767114:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701866188456 time 701879222148 state 0x0 ref 1 00000100:00100000:3.0:1789760771.767131:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701879482564 time 701879222148 state 0x1 ref 1 00000100:00100000:3.0:1789760771.767163:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701879482564 time 701891769617 state 0x0 ref 1 00000100:00100000:3.0:1789760771.767176:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ece300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503726720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760771.767221:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74870784, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23b50 00010000:00010000:3.0:1789760771.767230:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083efbda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.767246:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760771.767250:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.767257:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083efbda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74870784->74874879] (req 74870784->74874879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.767270:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083efbda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74870784->74874879] (req 74870784->74874879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.773957:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985586 is committed 00000100:00100000:1.0:1789760771.780373:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503726848 00000100:00100000:1.0:1789760771.780384:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503726848 00000100:00100000:1.0:1789760771.780472:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701879482564 time 701891769617 state 0x0 ref 1 00000100:00100000:1.0:1789760771.780488:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701892030033 time 701891769617 state 0x1 ref 1 00000100:00100000:1.0:1789760771.780525:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701892030033 time 701905131910 state 0x0 ref 1 00000100:00100000:1.0:1789760771.780538:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503726848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.780587:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74874880, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x23b58 00010000:00010000:1.0:1789760771.780596:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780611:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760771.780616:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.780622:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780635:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.785135:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985587 is committed 00002000:00010000:1.0:1789760771.785576:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74874880 -> 74878976 00002000:00010000:1.0:1789760771.785581:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:1.0:1789760771.785584:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00002000:00010000:1.0:1789760771.785586:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146264 -> 146272 00010000:00010000:1.0:1789760771.785599:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.785612:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.785621:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.785628:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.785633:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.785650:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.785671:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.785684:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbe1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74874880->74878975] (req 74874880->74878975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.785999:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503726848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5457us (6175us total) trans 4294985587 rc 0/0 00000100:00100000:1.0:1789760771.786010:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18291 00000100:00100000:1.0:1789760771.786020:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701892030033 time 701905131910 state 0x2 ref 0 00000100:00100000:0.0:1789760771.792169:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760771.795089:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606016 00000100:00100000:1.0:1789760771.795097:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606016 00000100:00100000:1.0:1789760771.795146:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:1.0:1789760771.795155:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523606016:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760771.795190:0:6517:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:1.0:1789760771.795249:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523606016:12345-0@lo:41:osp-pre-0-1.0 Request processed in 94us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760771.795256:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000001:00080000:0.0:1789760771.803577:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985588 is committed 00000100:00100000:1.0:1789760771.810314:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503727104 00000100:00100000:1.0:1789760771.810325:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503727104 00000100:00100000:1.0:1789760771.810463:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701905392326 time 701917041738 state 0x0 ref 1 00000100:00100000:1.0:1789760771.810478:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701917302154 time 701917041738 state 0x1 ref 1 00000100:00100000:1.0:1789760771.810560:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701917302154 time 701935166258 state 0x0 ref 1 00000100:00100000:1.0:1789760771.810574:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503727104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.810625:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74883072, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23b68 00010000:00010000:1.0:1789760771.810634:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.810661:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760771.810666:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.810672:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.810695:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.819922:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985589 is committed 00002000:00010000:1.0:1789760771.820091:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74883072 -> 74887168 00002000:00010000:1.0:1789760771.820096:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146280 -> 146288 00010000:00010000:1.0:1789760771.820110:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.820121:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.820130:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.820139:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.820143:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.820156:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.820176:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.820183:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74883072->74887167] (req 74883072->74887167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.820272:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503727104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9695us (10058us total) trans 4294985589 rc 0/0 00000100:00100000:1.0:1789760771.820283:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18293 00000100:00100000:1.0:1789760771.820292:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701917302154 time 701935166258 state 0x2 ref 0 00000100:00100000:3.0:1789760771.826083:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.826817:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503727232 00000100:00100000:0.0:1789760771.826828:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503727232 00000100:00100000:0.0:1789760771.826886:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701917302154 time 701935166258 state 0x0 ref 1 00000100:00100000:0.0:1789760771.826902:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701935426674 time 701935166258 state 0x1 ref 1 00000100:00100000:0.0:1789760771.826938:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701935426674 time 701951545268 state 0x0 ref 1 00000100:00100000:0.0:1789760771.826951:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503727232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.826999:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74887168, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23b70 00010000:00010000:0.0:1789760771.827008:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.827023:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.827027:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.827033:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.827046:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.834048:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985590 is committed 00002000:00010000:0.0:1789760771.839750:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74887168 -> 74891264 00002000:00010000:0.0:1789760771.839762:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146288 -> 146296 00010000:00010000:0.0:1789760771.839783:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.839802:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.839811:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.839818:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.839823:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.839831:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.839851:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.839859:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efbf6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74887168->74891263] (req 74887168->74891263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.840355:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503727232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13400us (14275us total) trans 4294985590 rc 0/0 00000100:00100000:0.0:1789760771.840366:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18294 00000100:00100000:0.0:1789760771.840376:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701935426674 time 701951545268 state 0x2 ref 0 00000100:00100000:0.0:1789760771.843068:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760771.843159:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503727360 00000100:00100000:1.0:1789760771.843169:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503727360 00000100:00100000:1.0:1789760771.843223:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701935426674 time 701951545268 state 0x0 ref 1 00000100:00100000:1.0:1789760771.843238:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701951805684 time 701951545268 state 0x1 ref 1 00000100:00100000:1.0:1789760771.843269:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701951805684 time 701967875540 state 0x0 ref 1 00000100:00100000:1.0:1789760771.843281:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503727360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.843326:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74891264, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23b78 00010000:00010000:1.0:1789760771.843336:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.843351:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760771.843355:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.843361:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.843495:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.850157:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985591 is committed 00002000:00010000:1.0:1789760771.850301:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74891264 -> 74895360 00002000:00010000:1.0:1789760771.850306:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146296 -> 146304 00010000:00010000:1.0:1789760771.850318:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.850331:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.850341:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.850348:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.850353:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.850362:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.850475:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.850485:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efbfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74891264->74895359] (req 74891264->74895359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.850579:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503727360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7294us (7514us total) trans 4294985591 rc 0/0 00000100:00100000:1.0:1789760771.850588:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18295 00000100:00100000:1.0:1789760771.850598:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701951805684 time 701967875540 state 0x2 ref 0 00000100:00100000:0.0:1789760771.860121:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503727488 00000100:00100000:0.0:1789760771.860131:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503727488 00000100:00100000:0.0:1789760771.860198:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701951805684 time 701967875540 state 0x0 ref 1 00000100:00100000:0.0:1789760771.860213:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701968135956 time 701967875540 state 0x1 ref 1 00000100:00100000:0.0:1789760771.860241:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701968135956 time 701984847148 state 0x0 ref 1 00000100:00100000:0.0:1789760771.860254:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503727488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.860305:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74895360, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23b80 00010000:00010000:0.0:1789760771.860314:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efc04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.860330:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760771.860335:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.860341:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efc04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74895360->74899455] (req 74895360->74899455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.860354:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efc04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74895360->74899455] (req 74895360->74899455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.867270:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985592 is committed 00000100:00100000:3.0:1789760771.873549:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:1.0:1789760771.884895:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74899456 -> 74903552 00002000:00010000:1.0:1789760771.884901:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146312 -> 146320 00010000:00010000:1.0:1789760771.884916:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083efc0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74899456->74903551] (req 74899456->74903551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.884928:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083efc0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74899456->74903551] (req 74899456->74903551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.884937:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083efc0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74899456->74903551] (req 74899456->74903551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.884944:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760771.884949:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083efc0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74899456->74903551] (req 74899456->74903551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.884963:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083efc0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74899456->74903551] (req 74899456->74903551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.884985:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083efc0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74899456->74903551] (req 74899456->74903551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.884993:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083efc0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74899456->74903551] (req 74899456->74903551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.885085:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503727616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10982us (11551us total) trans 4294985593 rc 0/0 00000100:00100000:1.0:1789760771.885095:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18297 00000100:00100000:1.0:1789760771.885104:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701985107564 time 701998693409 state 0x2 ref 0 00000100:00100000:0.0:1789760771.894120:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760771.903593:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985594 is committed 00000100:00100000:1.0:1789760771.919486:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503727872 00000100:00100000:1.0:1789760771.919496:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503727872 00000100:00100000:1.0:1789760771.919547:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 701998953825 time 702019523759 state 0x0 ref 1 00000100:00100000:1.0:1789760771.919560:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702019784175 time 702019523759 state 0x1 ref 1 00000100:00100000:1.0:1789760771.919685:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702019784175 time 702044283699 state 0x0 ref 1 00000100:00100000:1.0:1789760771.919902:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503727872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760771.919945:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74907648, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23b98 00010000:00010000:1.0:1789760771.919954:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.919968:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760771.919972:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.919979:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.919993:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.929434:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985595 is committed 00002000:00010000:0.0:1789760771.929586:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74907648 -> 74911744 00002000:00010000:0.0:1789760771.929591:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146328 -> 146336 00010000:00010000:0.0:1789760771.929604:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.929621:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.929628:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.929635:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.929639:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.929656:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.929675:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.929877:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efc19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74907648->74911743] (req 74907648->74911743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.929977:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503727872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10073us (10714us total) trans 4294985595 rc 0/0 00000100:00100000:0.0:1789760771.929988:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18299 00000100:00100000:0.0:1789760771.929998:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702019784175 time 702044283699 state 0x2 ref 0 00000100:00100000:3.0:1789760771.932523:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.932598:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503728000 00000100:00100000:0.0:1789760771.932608:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503728000 00000100:00100000:0.0:1789760771.932682:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702019784175 time 702044283699 state 0x0 ref 1 00000100:00100000:0.0:1789760771.932698:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702044544115 time 702044283699 state 0x1 ref 1 00000100:00100000:0.0:1789760771.932731:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702044544115 time 702057337465 state 0x0 ref 1 00000100:00100000:0.0:1789760771.932745:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503728000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.932794:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74911744, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23ba0 00010000:00010000:0.0:1789760771.932805:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.932831:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760771.932836:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.932843:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.932862:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.942829:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985596 is committed 00002000:00010000:0.0:1789760771.943598:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74911744 -> 74915840 00002000:00010000:0.0:1789760771.943604:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146336 -> 146344 00010000:00010000:0.0:1789760771.943618:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.943630:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.943661:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.943668:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.943673:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.943682:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.943701:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.943708:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74911744->74915839] (req 74911744->74915839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.943805:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503728000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11058us (11284us total) trans 4294985596 rc 0/0 00000100:00100000:0.0:1789760771.943815:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18300 00000100:00100000:0.0:1789760771.943826:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702044544115 time 702057337465 state 0x2 ref 0 00000100:00100000:1.0:1789760771.957256:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760771.976299:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985597 is committed 00000100:00100000:0.0:1789760771.980754:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503728256 00000100:00100000:0.0:1789760771.980764:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503728256 00000100:00100000:0.0:1789760771.980827:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702057597881 time 702083717820 state 0x0 ref 1 00000100:00100000:0.0:1789760771.980843:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702083978236 time 702083717820 state 0x1 ref 1 00000100:00100000:0.0:1789760771.980874:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702083978236 time 702105481232 state 0x0 ref 1 00000100:00100000:0.0:1789760771.980887:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503728256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.980942:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74919936, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23bb0 00010000:00010000:0.0:1789760771.980953:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.980969:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760771.980973:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.980979:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.980993:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.988821:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985598 is committed 00002000:00010000:0.0:1789760771.991738:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74919936 -> 74924032 00002000:00010000:0.0:1789760771.991748:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146352 -> 146360 00010000:00010000:0.0:1789760771.991761:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.991774:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.991782:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.991795:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760771.991800:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.991808:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.991828:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.991836:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74919936->74924031] (req 74919936->74924031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.991973:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503728256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11083us (11475us total) trans 4294985598 rc 0/0 00000100:00100000:0.0:1789760771.991984:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18302 00000100:00100000:0.0:1789760771.992003:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702083978236 time 702105481232 state 0x2 ref 0 00000100:00100000:3.0:1789760771.999551:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760771.999662:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503728384 00000100:00100000:0.0:1789760771.999672:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503728384 00000100:00100000:0.0:1789760771.999735:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702083978236 time 702105481232 state 0x0 ref 1 00000100:00100000:0.0:1789760771.999750:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702105741648 time 702105481232 state 0x1 ref 1 00000100:00100000:0.0:1789760771.999793:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702105741648 time 702124399600 state 0x0 ref 1 00000100:00100000:0.0:1789760771.999809:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503728384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760771.999853:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74924032, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23bb8 00010000:00010000:0.0:1789760771.999862:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.999877:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760771.999882:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.999888:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.999901:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.007710:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985599 is committed 00002000:00010000:0.0:1789760772.007893:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74924032 -> 74928128 00002000:00010000:0.0:1789760772.007898:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146360 -> 146368 00010000:00010000:0.0:1789760772.007911:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.007923:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.007933:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.007941:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.007947:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.007955:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.007973:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.007980:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efc35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74924032->74928127] (req 74924032->74928127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.008089:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503728384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8278us (8544us total) trans 4294985599 rc 0/0 00000100:00100000:0.0:1789760772.008100:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18303 00000100:00100000:0.0:1789760772.008110:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702105741648 time 702124399600 state 0x2 ref 0 00000100:00100000:0.0:1789760772.022659:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00080000:1.0:1789760772.022912:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760772.029489:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985600 is committed 00000100:00100000:1.0:1789760772.034966:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503728640 00000100:00100000:1.0:1789760772.034976:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503728640 00000100:00100000:1.0:1789760772.035096:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702124660016 time 702147570247 state 0x0 ref 1 00000100:00100000:1.0:1789760772.035112:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702147830663 time 702147570247 state 0x1 ref 1 00000100:00100000:1.0:1789760772.035140:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702147830663 time 702159746955 state 0x0 ref 1 00000100:00100000:1.0:1789760772.035153:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503728640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.035199:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74932224, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23bc8 00010000:00010000:1.0:1789760772.035209:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.035223:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.035227:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.035234:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74932224->74936319] (req 74932224->74936319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.035246:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74932224->74936319] (req 74932224->74936319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.042249:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985601 is committed 00000100:00100000:3.0:1789760772.051533:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760772.052834:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503728768 00000100:00100000:0.0:1789760772.052846:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503728768 00000100:00100000:0.0:1789760772.052905:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702147830663 time 702159746955 state 0x0 ref 1 00000100:00100000:0.0:1789760772.052921:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702160007371 time 702159746955 state 0x1 ref 1 00000100:00100000:0.0:1789760772.052946:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702160007371 time 702177552660 state 0x0 ref 1 00000100:00100000:0.0:1789760772.052958:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc1b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503728768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.053009:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74936320, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23bd0 00010000:00010000:0.0:1789760772.053018:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.053034:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.053038:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.053045:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.053059:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.060873:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985602 is committed 00002000:00010000:0.0:1789760772.061061:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74936320 -> 74940416 00002000:00010000:0.0:1789760772.061066:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146384 -> 146392 00010000:00010000:0.0:1789760772.061079:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.061091:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.061100:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.061108:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.061112:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.061126:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.061144:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.061152:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74936320->74940415] (req 74936320->74940415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.061311:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc1b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503728768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8338us (9771us total) trans 4294985602 rc 0/0 00000100:00100000:0.0:1789760772.061322:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18306 00000100:00100000:0.0:1789760772.061331:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702160007371 time 702177552660 state 0x2 ref 0 00000100:00100000:0.0:1789760772.067594:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.067693:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503728896 00000100:00100000:1.0:1789760772.067704:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503728896 00000100:00100000:1.0:1789760772.067756:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702160007371 time 702177552660 state 0x0 ref 1 00000100:00100000:1.0:1789760772.067771:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702177813076 time 702177552660 state 0x1 ref 1 00000100:00100000:1.0:1789760772.067804:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702177813076 time 702192411075 state 0x0 ref 1 00000100:00100000:1.0:1789760772.067817:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503728896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.067864:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74940416, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23bd8 00010000:00010000:1.0:1789760772.067872:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.067888:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.067893:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.067899:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.067914:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.078603:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985603 is committed 00002000:00010000:1.0:1789760772.079016:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74940416 -> 74944512 00002000:00010000:1.0:1789760772.079022:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146392 -> 146400 00010000:00010000:1.0:1789760772.079037:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.079050:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.079058:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.079065:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.079070:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.079078:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.079095:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.079108:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74940416->74944511] (req 74940416->74944511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.079205:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503728896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11385us (11616us total) trans 4294985603 rc 0/0 00000100:00100000:1.0:1789760772.079214:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18307 00000100:00100000:1.0:1789760772.079224:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702177813076 time 702192411075 state 0x2 ref 0 00000100:00100000:1.0:1789760772.087908:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503729024 00000100:00100000:1.0:1789760772.087918:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503729024 00000100:00100000:1.0:1789760772.087969:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702177813076 time 702192411075 state 0x0 ref 1 00000100:00100000:1.0:1789760772.087984:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702192671491 time 702192411075 state 0x1 ref 1 00000100:00100000:1.0:1789760772.088013:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702192671491 time 702212619659 state 0x0 ref 1 00000100:00100000:1.0:1789760772.088024:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503729024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.088067:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74944512, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23be0 00010000:00010000:1.0:1789760772.088075:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.088090:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.088094:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.088100:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.088118:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.094719:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985604 is committed 00002000:00010000:1.0:1789760772.094841:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74944512 -> 74948608 00002000:00010000:1.0:1789760772.094847:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146400 -> 146408 00010000:00010000:1.0:1789760772.094860:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.094873:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.094882:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.095518:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.095528:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.095542:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.095564:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.095572:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74944512->74948607] (req 74944512->74948607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.095735:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503729024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7707us (7936us total) trans 4294985604 rc 0/0 00000100:00100000:1.0:1789760772.095746:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18308 00000100:00100000:1.0:1789760772.095756:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702192671491 time 702212619659 state 0x2 ref 0 00000100:00100000:3.0:1789760772.102566:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.113611:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985605 is committed 00000100:00100000:0.0:1789760772.126362:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.127343:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702231668872 time 702251946096 state 0x0 ref 1 00000100:00100000:1.0:1789760772.127362:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503729280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.127445:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74952704, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23bf0 00010000:00010000:1.0:1789760772.127454:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.127469:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.127473:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.127480:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.127493:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.135130:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985606 is committed 00002000:00010000:1.0:1789760772.135261:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74952704 -> 74956800 00002000:00010000:1.0:1789760772.135267:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146416 -> 146424 00010000:00010000:1.0:1789760772.135281:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.135299:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.135308:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.135315:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.135320:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.135328:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.135348:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.135355:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083efc66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74952704->74956799] (req 74952704->74956799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.135515:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503729280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8153us (9157us total) trans 4294985606 rc 0/0 00000100:00100000:1.0:1789760772.135525:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18310 00000100:00100000:1.0:1789760772.135535:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702231668872 time 702251946096 state 0x2 ref 0 00000100:00100000:1.0:1789760772.141495:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760772.141582:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503729408 00000100:00100000:0.0:1789760772.141591:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503729408 00000100:00100000:0.0:1789760772.141657:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702231668872 time 702251946096 state 0x0 ref 1 00000100:00100000:0.0:1789760772.141674:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702252206512 time 702251946096 state 0x1 ref 1 00000100:00100000:0.0:1789760772.141704:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702252206512 time 702266311361 state 0x0 ref 1 00000100:00100000:0.0:1789760772.141717:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503729408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.141764:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74956800, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23bf8 00010000:00010000:0.0:1789760772.141773:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.141789:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.141793:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.141799:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.141813:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.153550:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985607 is committed 00002000:00010000:0.0:1789760772.153690:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74956800 -> 74960896 00002000:00010000:0.0:1789760772.153695:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146424 -> 146432 00010000:00010000:0.0:1789760772.153707:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.153721:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.153731:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.153738:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.153743:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.153758:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.153778:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.153786:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083efc6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74956800->74960895] (req 74956800->74960895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.154161:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503729408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12168us (12400us total) trans 4294985607 rc 0/0 00000100:00100000:0.0:1789760772.154177:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18311 00000100:00100000:0.0:1789760772.154187:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702252206512 time 702266311361 state 0x2 ref 0 00000100:00100000:3.0:1789760772.157707:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.165899:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985608 is committed 00000100:00100000:1.0:1789760772.177726:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503729664 00000100:00100000:1.0:1789760772.177736:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503729664 00000100:00100000:1.0:1789760772.177789:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702266571777 time 702282591842 state 0x0 ref 1 00000100:00100000:1.0:1789760772.177805:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702282852258 time 702282591842 state 0x1 ref 1 00000100:00100000:1.0:1789760772.177841:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702282852258 time 702302447784 state 0x0 ref 1 00000100:00100000:1.0:1789760772.177852:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503729664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.177901:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74964992, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c08 00010000:00010000:1.0:1789760772.177908:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.177922:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.177927:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.177933:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.177945:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.188449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985609 is committed 00002000:00010000:1.0:1789760772.188595:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74964992 -> 74969088 00002000:00010000:1.0:1789760772.188600:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146440 -> 146448 00010000:00010000:1.0:1789760772.188611:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.188624:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.188634:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.188654:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.188659:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.188668:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.188688:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.188696:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083efc7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74964992->74969087] (req 74964992->74969087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.188791:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503729664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10936us (11176us total) trans 4294985609 rc 0/0 00000100:00100000:1.0:1789760772.188801:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18313 00000100:00100000:1.0:1789760772.188810:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702282852258 time 702302447784 state 0x2 ref 0 00000100:00100000:1.0:1789760772.196725:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760772.207080:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985610 is committed 00000100:00100000:3.0:1789760772.212959:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760772.213106:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503729920 00000100:00100000:0.0:1789760772.213117:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503729920 00000100:00100000:0.0:1789760772.213174:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702302708200 time 702324611136 state 0x0 ref 1 00000100:00100000:0.0:1789760772.213189:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702324871552 time 702324611136 state 0x1 ref 1 00000100:00100000:0.0:1789760772.213228:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702324871552 time 702337833919 state 0x0 ref 1 00000100:00100000:0.0:1789760772.213241:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503729920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.213288:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74973184, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c18 00010000:00010000:0.0:1789760772.213298:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.213315:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.213320:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.213326:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.213340:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.223463:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985611 is committed 00002000:00010000:0.0:1789760772.224011:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74973184 -> 74977280 00002000:00010000:0.0:1789760772.224016:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146456 -> 146464 00010000:00010000:0.0:1789760772.224029:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.224041:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.224049:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.224056:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.224060:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.224069:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.224092:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.224100:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083efc89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74973184->74977279] (req 74973184->74977279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.224194:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503729920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10951us (11240us total) trans 4294985611 rc 0/0 00000100:00100000:0.0:1789760772.224205:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18315 00000100:00100000:0.0:1789760772.224214:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702324871552 time 702337833919 state 0x2 ref 0 00000100:00100000:0.0:1789760772.231837:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.244323:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985612 is committed 00000100:00100000:1.0:1789760772.252917:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.261851:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985613 is committed 00000100:00100000:0.0:1789760772.267807:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.268047:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503730304 00000100:00100000:1.0:1789760772.268059:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503730304 00000100:00100000:1.0:1789760772.268122:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702356940435 time 702377727883 state 0x0 ref 1 00000100:00100000:1.0:1789760772.268136:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702377988299 time 702377727883 state 0x1 ref 1 00000100:00100000:1.0:1789760772.268207:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702377988299 time 702392812447 state 0x0 ref 1 00000100:00100000:1.0:1789760772.268220:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503730304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.268268:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74985472, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c30 00010000:00010000:1.0:1789760772.268277:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083efc9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.268293:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.268297:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.268304:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083efc9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74985472->74989567] (req 74985472->74989567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.268317:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083efc9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74985472->74989567] (req 74985472->74989567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.276013:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985614 is committed 00000100:00100000:0.0:1789760772.280015:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.280071:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503730432 00000100:00100000:1.0:1789760772.280080:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503730432 00000100:00100000:1.0:1789760772.280135:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702377988299 time 702392812447 state 0x0 ref 1 00000100:00100000:1.0:1789760772.280159:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702393072863 time 702392812447 state 0x1 ref 1 00000100:00100000:1.0:1789760772.280221:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702393072863 time 702404827351 state 0x0 ref 1 00000100:00100000:1.0:1789760772.280233:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503730432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.280278:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 74989568, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c38 00010000:00010000:1.0:1789760772.280288:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.280303:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.280307:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.280312:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.280324:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.291374:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985615 is committed 00002000:00010000:0.0:1789760772.291561:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 74989568 -> 74993664 00002000:00010000:0.0:1789760772.291566:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146488 -> 146496 00010000:00010000:0.0:1789760772.291579:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.291592:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.291600:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.291610:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.291614:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.291622:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.291650:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.291659:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efca5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [74989568->74993663] (req 74989568->74993663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.291764:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503730432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11528us (11749us total) trans 4294985615 rc 0/0 00000100:00100000:0.0:1789760772.291776:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18319 00000100:00100000:0.0:1789760772.291786:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702393072863 time 702404827351 state 0x2 ref 0 00000100:00100000:1.0:1789760772.294973:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760772.302445:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985616 is committed 00000100:00100000:3.0:1789760772.307151:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.317332:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985617 is committed 00000100:00100000:0.0:1789760772.330625:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.332278:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503730816 00000100:00100000:1.0:1789760772.332300:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503730816 00000100:00100000:1.0:1789760772.332361:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702420649057 time 702432207595 state 0x0 ref 1 00000100:00100000:1.0:1789760772.332377:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702432468011 time 702432207595 state 0x1 ref 1 00000100:00100000:1.0:1789760772.332486:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702432468011 time 702457092523 state 0x0 ref 1 00000100:00100000:1.0:1789760772.332503:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503730816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.332548:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75001856, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c50 00010000:00010000:1.0:1789760772.332557:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.332573:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.332577:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.332583:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.332597:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.341028:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985618 is committed 00002000:00010000:1.0:1789760772.341235:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75001856 -> 75005952 00002000:00010000:1.0:1789760772.341241:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146512 -> 146520 00010000:00010000:1.0:1789760772.341255:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.341268:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.341277:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.341285:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.341290:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.341300:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.341321:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.341329:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efcba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75001856->75005951] (req 75001856->75005951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.341888:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503730816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9379us (11265us total) trans 4294985618 rc 0/0 00000100:00100000:1.0:1789760772.341901:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18322 00000100:00100000:1.0:1789760772.341911:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702432468011 time 702457092523 state 0x2 ref 0 00000100:00100000:1.0:1789760772.351235:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760772.351342:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503730944 00000100:00100000:0.0:1789760772.351352:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503730944 00000100:00100000:0.0:1789760772.351453:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702432468011 time 702457092523 state 0x0 ref 1 00000100:00100000:0.0:1789760772.351467:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702457352939 time 702457092523 state 0x1 ref 1 00000100:00100000:0.0:1789760772.351497:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702457352939 time 702476104414 state 0x0 ref 1 00000100:00100000:0.0:1789760772.351510:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503730944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.351552:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75005952, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c58 00010000:00010000:0.0:1789760772.351561:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.351576:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.351580:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.351598:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.351612:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.361380:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985619 is committed 00002000:00010000:0.0:1789760772.361512:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75005952 -> 75010048 00002000:00010000:0.0:1789760772.361517:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146520 -> 146528 00010000:00010000:0.0:1789760772.361530:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.361543:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.361553:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.361561:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.361566:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.361576:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.361595:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.361603:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efcc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75005952->75010047] (req 75005952->75010047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.361721:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503730944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10208us (10489us total) trans 4294985619 rc 0/0 00000100:00100000:0.0:1789760772.361732:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18323 00000100:00100000:0.0:1789760772.361741:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702457352939 time 702476104414 state 0x2 ref 0 00000100:00100000:0.0:1789760772.365258:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.372666:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985620 is committed 00000100:00100000:1.0:1789760772.377376:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503731200 00000100:00100000:1.0:1789760772.377385:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503731200 00000100:00100000:1.0:1789760772.377479:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702476364830 time 702490542615 state 0x0 ref 1 00000100:00100000:1.0:1789760772.377498:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702490803031 time 702490542615 state 0x1 ref 1 00000100:00100000:1.0:1789760772.377522:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702490803031 time 702502128697 state 0x0 ref 1 00000100:00100000:1.0:1789760772.377535:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.377584:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75014144, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c68 00010000:00010000:1.0:1789760772.377593:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.377611:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.377615:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.377622:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.377635:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.387218:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985621 is committed 00002000:00010000:1.0:1789760772.387357:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75014144 -> 75018240 00002000:00010000:1.0:1789760772.387372:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146536 -> 146544 00010000:00010000:1.0:1789760772.387386:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.387512:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.387522:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.387529:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.387535:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.387551:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.387571:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.387579:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efccf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75014144->75018239] (req 75014144->75018239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.387720:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10181us (10415us total) trans 4294985621 rc 0/0 00000100:00100000:1.0:1789760772.387731:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18325 00000100:00100000:1.0:1789760772.387741:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702490803031 time 702502128697 state 0x2 ref 0 00000100:00100000:1.0:1789760772.394611:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760772.404062:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985622 is committed 00000100:00100000:0.0:1789760772.410921:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.410998:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503731456 00000100:00100000:1.0:1789760772.411009:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503731456 00000100:00100000:1.0:1789760772.411063:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702502389113 time 702519428792 state 0x0 ref 1 00000100:00100000:1.0:1789760772.411076:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702519689208 time 702519428792 state 0x1 ref 1 00000100:00100000:1.0:1789760772.411109:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702519689208 time 702535715751 state 0x0 ref 1 00000100:00100000:1.0:1789760772.411120:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.411164:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75022336, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c78 00010000:00010000:1.0:1789760772.411173:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.411187:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.411191:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.411197:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.411208:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.416858:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985623 is committed 00002000:00010000:1.0:1789760772.417061:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75022336 -> 75026432 00002000:00010000:1.0:1789760772.417066:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146552 -> 146560 00010000:00010000:1.0:1789760772.417079:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.417090:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.417098:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.417104:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.417108:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.417116:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.417137:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.417145:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efcdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75022336->75026431] (req 75022336->75026431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.417284:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6161us (6368us total) trans 4294985623 rc 0/0 00000100:00100000:1.0:1789760772.417294:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18327 00000100:00100000:1.0:1789760772.417305:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702519689208 time 702535715751 state 0x2 ref 0 00000100:00100000:0.0:1789760772.423108:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760772.431743:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985624 is committed 00000100:00100000:1.0:1789760772.436083:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503731712 00000100:00100000:1.0:1789760772.436093:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503731712 00000100:00100000:1.0:1789760772.436148:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702535976167 time 702547917806 state 0x0 ref 1 00000100:00100000:1.0:1789760772.436162:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702548178222 time 702547917806 state 0x1 ref 1 00000100:00100000:1.0:1789760772.436192:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702548178222 time 702560798909 state 0x0 ref 1 00000100:00100000:1.0:1789760772.436204:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.436252:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75030528, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c88 00010000:00010000:1.0:1789760772.436261:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.436276:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.436281:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.436287:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.436300:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.444835:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985625 is committed 00002000:00010000:1.0:1789760772.445828:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75030528 -> 75034624 00002000:00010000:1.0:1789760772.445835:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146568 -> 146576 00010000:00010000:1.0:1789760772.445848:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.445862:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.445871:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.445879:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.445887:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.445900:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.445919:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.445931:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efceb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75030528->75034623] (req 75030528->75034623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.446026:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9819us (10233us total) trans 4294985625 rc 0/0 00000100:00100000:1.0:1789760772.446035:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18329 00000100:00100000:1.0:1789760772.446045:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702548178222 time 702560798909 state 0x2 ref 0 00000100:00100000:0.0:1789760772.448861:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.449240:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503731840 00000100:00100000:1.0:1789760772.449250:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503731840 00000100:00100000:1.0:1789760772.449304:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702548178222 time 702560798909 state 0x0 ref 1 00000100:00100000:1.0:1789760772.449317:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702561059325 time 702560798909 state 0x1 ref 1 00000100:00100000:1.0:1789760772.449343:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702561059325 time 702573949925 state 0x0 ref 1 00000100:00100000:1.0:1789760772.449354:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503731840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.449429:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75034624, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c90 00010000:00010000:1.0:1789760772.449438:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efcf2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.449454:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.449460:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.449465:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efcf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75034624->75038719] (req 75034624->75038719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.449482:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efcf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75034624->75038719] (req 75034624->75038719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.457957:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985626 is committed 00000100:00100000:0.0:1789760772.462768:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.462940:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503731968 00000100:00100000:1.0:1789760772.462950:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503731968 00000100:00100000:1.0:1789760772.463003:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702561059325 time 702573949925 state 0x0 ref 1 00000100:00100000:1.0:1789760772.463016:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702574210341 time 702573949925 state 0x1 ref 1 00000100:00100000:1.0:1789760772.463047:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702574210341 time 702587654620 state 0x0 ref 1 00000100:00100000:1.0:1789760772.463059:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.463103:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75038720, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23c98 00010000:00010000:1.0:1789760772.463111:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.463125:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.463130:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.463136:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.463149:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.470841:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985627 is committed 00002000:00010000:0.0:1789760772.470954:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75038720 -> 75042816 00002000:00010000:0.0:1789760772.470960:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146584 -> 146592 00010000:00010000:0.0:1789760772.470974:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.470987:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.470996:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.471003:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.471008:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.471018:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.471037:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.471044:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083efcf9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75038720->75042815] (req 75038720->75042815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.471142:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503731968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8079us (8378us total) trans 4294985627 rc 0/0 00000100:00100000:0.0:1789760772.471164:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18331 00000100:00100000:0.0:1789760772.471174:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702574210341 time 702587654620 state 0x2 ref 0 00000100:00100000:1.0:1789760772.476672:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760772.484571:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985628 is committed 00000100:00100000:3.0:1789760772.491869:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:1.0:1789760772.498140:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75046912 -> 75051008 00002000:00010000:1.0:1789760772.498145:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146600 -> 146608 00010000:00010000:1.0:1789760772.498158:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efd07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75046912->75051007] (req 75046912->75051007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.498172:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efd07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75046912->75051007] (req 75046912->75051007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.498181:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efd07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75046912->75051007] (req 75046912->75051007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.498189:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.498194:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efd07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75046912->75051007] (req 75046912->75051007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.498203:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efd07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75046912->75051007] (req 75046912->75051007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.498221:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efd07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75046912->75051007] (req 75046912->75051007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.498228:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083efd07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75046912->75051007] (req 75046912->75051007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.498603:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503732224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6397us (6737us total) trans 4294985629 rc 0/0 00000100:00100000:1.0:1789760772.498614:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18333 00000100:00100000:1.0:1789760772.498624:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702602780822 time 702616796429 state 0x2 ref 0 00000100:00100000:0.0:1789760772.503112:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.503277:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503732352 00000100:00100000:1.0:1789760772.503288:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503732352 00000100:00100000:1.0:1789760772.503549:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702602780822 time 702616796429 state 0x0 ref 1 00000100:00100000:1.0:1789760772.503563:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702617056845 time 702616796429 state 0x1 ref 1 00000100:00100000:1.0:1789760772.503594:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702617056845 time 702628200616 state 0x0 ref 1 00000100:00100000:1.0:1789760772.503607:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503732352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.503763:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75051008, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23cb0 00010000:00010000:1.0:1789760772.503772:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.503787:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.503791:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.503797:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.503810:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.508715:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75051008 -> 75055104 00002000:00010000:1.0:1789760772.508720:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146608 -> 146616 00010000:00010000:1.0:1789760772.508733:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.508746:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.508755:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.508762:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.508766:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.508775:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.508794:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.508801:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75051008->75055103] (req 75051008->75055103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.508898:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503732352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5288us (5803us total) trans 4294985630 rc 0/0 00000100:00100000:1.0:1789760772.508909:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18334 00000100:00100000:1.0:1789760772.508918:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702617056845 time 702628200616 state 0x2 ref 0 00000100:00100000:0.0:1789760772.517038:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503732480 00000100:00100000:0.0:1789760772.517047:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503732480 00000100:00100000:0.0:1789760772.517108:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702617056845 time 702628200616 state 0x0 ref 1 00000100:00100000:0.0:1789760772.517123:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702628461032 time 702628200616 state 0x1 ref 1 00000100:00100000:0.0:1789760772.517194:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702628461032 time 702641801032 state 0x0 ref 1 00000100:00100000:0.0:1789760772.517206:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503732480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.517251:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75055104, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23cb8 00010000:00010000:0.0:1789760772.517259:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.517273:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.517277:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.517283:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.517316:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.522057:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75055104 -> 75059200 00002000:00010000:0.0:1789760772.522062:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146616 -> 146624 00010000:00010000:0.0:1789760772.522075:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.522092:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.522100:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.522108:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.522112:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.522121:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.522139:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.522146:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75055104->75059199] (req 75055104->75059199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.522271:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503732480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5061us (5338us total) trans 4294985631 rc 0/0 00000100:00100000:0.0:1789760772.522281:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18335 00000100:00100000:0.0:1789760772.522299:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702628461032 time 702641801032 state 0x2 ref 0 00000100:00100000:3.0:1789760772.530260:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760772.537357:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75059200 -> 75063296 00002000:00010000:0.0:1789760772.537370:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146624 -> 146632 00010000:00010000:0.0:1789760772.537383:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75059200->75063295] (req 75059200->75063295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.537422:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75059200->75063295] (req 75059200->75063295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.537427:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75059200->75063295] (req 75059200->75063295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.537432:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.537436:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75059200->75063295] (req 75059200->75063295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.537442:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75059200->75063295] (req 75059200->75063295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.537457:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75059200->75063295] (req 75059200->75063295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.537462:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75059200->75063295] (req 75059200->75063295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.537542:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503732608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6624us (7287us total) trans 4294985632 rc 0/0 00000100:00100000:0.0:1789760772.537552:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18336 00000100:00100000:0.0:1789760772.537561:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702642061448 time 702655505758 state 0x2 ref 0 00000100:00100000:1.0:1789760772.540578:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503732736 00000100:00100000:1.0:1789760772.540588:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503732736 00000100:00100000:1.0:1789760772.540703:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702642061448 time 702655505758 state 0x0 ref 1 00000100:00100000:1.0:1789760772.540720:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702655766174 time 702655505758 state 0x1 ref 1 00000100:00100000:1.0:1789760772.540968:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702655766174 time 702665388280 state 0x0 ref 1 00000100:00100000:1.0:1789760772.540987:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503732736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.541036:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75063296, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23cc8 00010000:00010000:1.0:1789760772.541045:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.541061:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.541065:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.541071:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.541084:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.545859:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985633 is committed 00002000:00010000:1.0:1789760772.545967:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75063296 -> 75067392 00002000:00010000:1.0:1789760772.545972:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146632 -> 146640 00010000:00010000:1.0:1789760772.545985:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.545998:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.546007:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.546014:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.546019:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.546028:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.546047:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.546054:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efd23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75063296->75067391] (req 75063296->75067391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.546176:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503732736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5188us (5922us total) trans 4294985633 rc 0/0 00000100:00100000:1.0:1789760772.546188:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18337 00000100:00100000:1.0:1789760772.546197:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702655766174 time 702665388280 state 0x2 ref 0 00000100:00100000:0.0:1789760772.552943:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.553015:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503732992 00000100:00100000:1.0:1789760772.553026:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503732992 00000100:00100000:1.0:1789760772.553090:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503733248 00000100:00100000:1.0:1789760772.553095:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503733248 00000100:00100000:1.0:1789760772.553113:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503733120 00000100:00100000:1.0:1789760772.553118:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503733120 00000100:00100000:1.0:1789760772.553139:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503733376 00000100:00100000:1.0:1789760772.553142:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503733376 00000100:00100000:1.0:1789760772.553164:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 951 00000100:00100000:1.0:1789760772.553175:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2390:x1876699503732992:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760772.553242:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 952 00000100:00100000:0.0:1789760772.553251:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503733248:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760772.553325:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 953 00000100:00100000:3.0:1789760772.553335:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503733120:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:1.0:1789760772.554033:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2390:x1876699503732992:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 858us (1100us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760772.554041:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 951 00000100:00100000:1.0:1789760772.554058:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 954 00000100:00100000:1.0:1789760772.554066:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503733376:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:1.0:1789760772.554095:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503733376:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 30us (1017us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760772.554101:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 954 00000100:00080000:1.0:1789760772.554119:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760772.554123:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760772.554136:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503733504 00000100:00100000:1.0:1789760772.554140:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503733504 00000100:00100000:1.0:1789760772.554175:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702655766174 time 702665388280 state 0x0 ref 1 00000100:00100000:1.0:1789760772.554187:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702665648696 time 702665388280 state 0x1 ref 1 00000100:00100000:1.0:1789760772.554209:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702665648696 time 702678815273 state 0x0 ref 1 00000100:00100000:1.0:1789760772.554219:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503733504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.554272:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75067392, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23cd0 00010000:00010000:1.0:1789760772.554281:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.554519:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.554525:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.554532:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.554546:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.556145:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2390:x1876699503733248:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 2886us (3148us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760772.556156:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 952 00000100:00100000:3.0:1789760772.556469:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503733120:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 3134us (3418us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.556478:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 953 00002000:00010000:1.0:1789760772.567254:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75067392 -> 75071488 00002000:00010000:1.0:1789760772.567260:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146640 -> 146648 00010000:00010000:1.0:1789760772.567275:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.567287:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.567297:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.567375:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.567380:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.567411:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.567432:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.567439:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75067392->75071487] (req 75067392->75071487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.567606:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503733504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13383us (14667us total) trans 4294985634 rc 0/0 00000100:00100000:1.0:1789760772.567617:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18338 00000100:00100000:1.0:1789760772.567627:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702665648696 time 702678815273 state 0x2 ref 0 00000100:00100000:1.0:1789760772.579338:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760772.579477:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503733632 00000100:00100000:0.0:1789760772.579487:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503733632 00000100:00100000:0.0:1789760772.579543:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702665648696 time 702678815273 state 0x0 ref 1 00000100:00100000:0.0:1789760772.579556:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702679075689 time 702678815273 state 0x1 ref 1 00000100:00100000:0.0:1789760772.579581:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702679075689 time 702704187966 state 0x0 ref 1 00000100:00100000:0.0:1789760772.579592:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503733632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.579654:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75071488, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23cd8 00010000:00010000:0.0:1789760772.579671:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.579689:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.579693:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.579699:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.579712:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.587900:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75071488 -> 75075584 00002000:00010000:0.0:1789760772.587906:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146648 -> 146656 00010000:00010000:0.0:1789760772.587919:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.587930:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.587938:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.587944:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.587948:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.587957:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.587976:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.587982:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efd31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75071488->75075583] (req 75071488->75075583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.588077:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503733632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8481us (8742us total) trans 4294985635 rc 0/0 00000100:00100000:0.0:1789760772.588088:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18339 00000100:00100000:0.0:1789760772.588097:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702679075689 time 702704187966 state 0x2 ref 0 00000100:00100000:1.0:1789760772.595069:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503733760 00000100:00100000:1.0:1789760772.595078:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503733760 00000100:00100000:1.0:1789760772.595131:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702679075689 time 702704187966 state 0x0 ref 1 00000100:00100000:1.0:1789760772.595145:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702704448382 time 702704187966 state 0x1 ref 1 00000100:00100000:1.0:1789760772.595212:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702704448382 time 702719818135 state 0x0 ref 1 00000100:00100000:1.0:1789760772.595225:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503733760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.595287:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75075584, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23ce0 00010000:00010000:1.0:1789760772.595296:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.595311:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.595316:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.595323:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.595338:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.602645:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75075584 -> 75079680 00002000:00010000:1.0:1789760772.602650:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146656 -> 146664 00010000:00010000:1.0:1789760772.602663:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.602753:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.602764:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.602772:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.602777:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.602792:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.602811:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.602818:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75075584->75079679] (req 75075584->75079679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.602948:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503733760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7720us (8025us total) trans 4294985636 rc 0/0 00000100:00100000:1.0:1789760772.602959:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18340 00000100:00100000:1.0:1789760772.602968:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702704448382 time 702719818135 state 0x2 ref 0 00000100:00100000:1.0:1789760772.607101:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:1.0:1789760772.614824:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75079680 -> 75083776 00002000:00010000:1.0:1789760772.614829:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146664 -> 146672 00010000:00010000:1.0:1789760772.614842:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efd3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75079680->75083775] (req 75079680->75083775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.614855:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efd3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75079680->75083775] (req 75079680->75083775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.614864:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efd3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75079680->75083775] (req 75079680->75083775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.614877:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.614882:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efd3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75079680->75083775] (req 75079680->75083775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.614891:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efd3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75079680->75083775] (req 75079680->75083775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.614910:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efd3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75079680->75083775] (req 75079680->75083775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.614918:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083efd3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75079680->75083775] (req 75079680->75083775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.615067:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503733888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7736us (7970us total) trans 4294985637 rc 0/0 00000100:00100000:1.0:1789760772.615078:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18341 00000100:00100000:1.0:1789760772.615089:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702720078551 time 702731921650 state 0x2 ref 0 00000100:00100000:0.0:1789760772.620649:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760772.626348:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985638 is committed 00002000:00010000:0.0:1789760772.626608:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75083776 -> 75087872 00002000:00010000:0.0:1789760772.626613:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146672 -> 146680 00010000:00010000:0.0:1789760772.626626:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75083776->75087871] (req 75083776->75087871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.626649:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75083776->75087871] (req 75083776->75087871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.626658:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75083776->75087871] (req 75083776->75087871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.626665:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.626670:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75083776->75087871] (req 75083776->75087871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.626688:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75083776->75087871] (req 75083776->75087871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.626705:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75083776->75087871] (req 75083776->75087871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.626713:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efd46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75083776->75087871] (req 75083776->75087871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.626800:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503734016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5935us (6167us total) trans 4294985638 rc 0/0 00000100:00100000:0.0:1789760772.626810:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18342 00000100:00100000:0.0:1789760772.626820:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702732182066 time 702745456843 state 0x2 ref 0 00010000:00010000:3.0:1789760772.626994:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760772.631540:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503734144 00000100:00100000:0.0:1789760772.631558:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503734144 00000100:00100000:0.0:1789760772.631616:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702732182066 time 702745456843 state 0x0 ref 1 00000100:00100000:0.0:1789760772.631633:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702745717259 time 702745456843 state 0x1 ref 1 00000100:00100000:0.0:1789760772.631725:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702745717259 time 702756331281 state 0x0 ref 1 00000100:00100000:0.0:1789760772.631738:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503734144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.631787:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75087872, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23cf8 00010000:00010000:0.0:1789760772.631797:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.631812:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.631816:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.631822:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.631836:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.636723:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985639 is committed 00002000:00010000:0.0:1789760772.636830:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75087872 -> 75091968 00002000:00010000:0.0:1789760772.636835:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146680 -> 146688 00010000:00010000:0.0:1789760772.636847:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.636858:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.636866:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.636873:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.636877:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.636885:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.636904:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.636912:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efd4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75087872->75091967] (req 75087872->75091967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.636999:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503734144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5255us (5508us total) trans 4294985639 rc 0/0 00000100:00100000:0.0:1789760772.637010:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18343 00000100:00100000:0.0:1789760772.637019:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702745717259 time 702756331281 state 0x2 ref 0 00000100:00100000:0.0:1789760772.640339:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.640443:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503734272 00000100:00100000:1.0:1789760772.640452:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503734272 00000100:00100000:1.0:1789760772.640507:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702745717259 time 702756331281 state 0x0 ref 1 00000100:00100000:1.0:1789760772.640522:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702756591697 time 702756331281 state 0x1 ref 1 00000100:00100000:1.0:1789760772.640624:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702756591697 time 702765229767 state 0x0 ref 1 00000100:00100000:1.0:1789760772.640649:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503734272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.640752:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75091968, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d00 00010000:00010000:1.0:1789760772.640761:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.640775:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.640779:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.640786:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.640800:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.649231:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985640 is committed 00002000:00010000:1.0:1789760772.656098:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75091968 -> 75096064 00002000:00010000:1.0:1789760772.656108:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146688 -> 146696 00010000:00010000:1.0:1789760772.656122:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.656136:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.656145:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.656159:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.656164:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.656173:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.656193:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.656202:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083efd54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75091968->75096063] (req 75091968->75096063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.657126:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503734272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16480us (16784us total) trans 4294985640 rc 0/0 00000100:00100000:1.0:1789760772.657138:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18344 00000100:00100000:1.0:1789760772.657148:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702756591697 time 702765229767 state 0x2 ref 0 00000100:00100000:1.0:1789760772.666006:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.677172:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985641 is committed 00000100:00100000:0.0:1789760772.685355:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503734528 00000100:00100000:0.0:1789760772.685364:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503734528 00000100:00100000:0.0:1789760772.685452:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702765490183 time 702790950832 state 0x0 ref 1 00000100:00100000:0.0:1789760772.685466:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702791211248 time 702790950832 state 0x1 ref 1 00000100:00100000:0.0:1789760772.685486:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702791211248 time 702810093970 state 0x0 ref 1 00000100:00100000:0.0:1789760772.685500:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503734528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.685562:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75100160, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d10 00010000:00010000:0.0:1789760772.685570:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.685585:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.685589:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.685595:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.685608:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.692970:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985642 is committed 00002000:00010000:0.0:1789760772.693101:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75100160 -> 75104256 00002000:00010000:0.0:1789760772.693106:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146704 -> 146712 00010000:00010000:0.0:1789760772.693118:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.693130:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.693139:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.693146:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.693150:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.693158:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.693176:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.693182:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75100160->75104255] (req 75100160->75104255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.693275:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503734528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7773us (8013us total) trans 4294985642 rc 0/0 00000100:00100000:0.0:1789760772.693285:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18346 00000100:00100000:0.0:1789760772.693295:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702791211248 time 702810093970 state 0x2 ref 0 00000100:00100000:0.0:1789760772.699303:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.699854:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503734656 00000100:00100000:1.0:1789760772.699864:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503734656 00000100:00100000:1.0:1789760772.699917:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702791211248 time 702810093970 state 0x0 ref 1 00000100:00100000:1.0:1789760772.699932:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702810354386 time 702810093970 state 0x1 ref 1 00000100:00100000:1.0:1789760772.699965:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702810354386 time 702824571699 state 0x0 ref 1 00000100:00100000:1.0:1789760772.699977:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503734656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.700024:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75104256, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d18 00010000:00010000:1.0:1789760772.700033:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.700048:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.700053:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.700059:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.700071:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.708000:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985643 is committed 00002000:00010000:1.0:1789760772.708128:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75104256 -> 75108352 00002000:00010000:1.0:1789760772.708133:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146712 -> 146720 00010000:00010000:1.0:1789760772.708147:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.708159:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.708168:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.708175:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.708180:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.708188:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.708205:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.708211:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75104256->75108351] (req 75104256->75108351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.708501:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503734656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8520us (9264us total) trans 4294985643 rc 0/0 00000100:00100000:1.0:1789760772.708512:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18347 00000100:00100000:1.0:1789760772.708522:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702810354386 time 702824571699 state 0x2 ref 0 00000100:00100000:1.0:1789760772.713665:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760772.713759:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503734784 00000100:00100000:0.0:1789760772.713770:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503734784 00000100:00100000:0.0:1789760772.713826:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702810354386 time 702824571699 state 0x0 ref 1 00000100:00100000:0.0:1789760772.713840:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702824832115 time 702824571699 state 0x1 ref 1 00000100:00100000:0.0:1789760772.713874:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702824832115 time 702838480914 state 0x0 ref 1 00000100:00100000:0.0:1789760772.713887:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503734784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.713933:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75108352, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d20 00010000:00010000:0.0:1789760772.713942:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713959:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.713963:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.713969:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.713982:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.719161:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985644 is committed 00002000:00010000:0.0:1789760772.719262:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75108352 -> 75112448 00002000:00010000:0.0:1789760772.719268:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146720 -> 146728 00010000:00010000:0.0:1789760772.719280:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.719297:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.719306:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.719313:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.719318:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.719327:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.719348:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.719355:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efd70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75108352->75112447] (req 75108352->75112447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.719483:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503734784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5593us (5823us total) trans 4294985644 rc 0/0 00000100:00100000:0.0:1789760772.719493:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18348 00000100:00100000:0.0:1789760772.719503:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702824832115 time 702838480914 state 0x2 ref 0 00000100:00100000:1.0:1789760772.722704:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503734912 00000100:00100000:1.0:1789760772.722713:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503734912 00000100:00100000:1.0:1789760772.722765:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702824832115 time 702838480914 state 0x0 ref 1 00000100:00100000:1.0:1789760772.722779:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702838741330 time 702838480914 state 0x1 ref 1 00000100:00100000:1.0:1789760772.722808:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702838741330 time 702847415628 state 0x0 ref 1 00000100:00100000:1.0:1789760772.722821:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503734912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.722872:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75112448, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d28 00010000:00010000:1.0:1789760772.722882:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.722898:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.722903:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.722909:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.722923:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.728185:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985645 is committed 00002000:00010000:1.0:1789760772.730513:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75112448 -> 75116544 00002000:00010000:1.0:1789760772.730522:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146728 -> 146736 00010000:00010000:1.0:1789760772.730537:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731902:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731922:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.732059:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.732065:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.732083:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.732105:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.732115:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75112448->75116543] (req 75112448->75116543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.732229:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503734912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9405us (9682us total) trans 4294985645 rc 0/0 00000100:00100000:1.0:1789760772.732349:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18349 00000100:00100000:1.0:1789760772.732359:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702838741330 time 702847415628 state 0x2 ref 0 00000100:00100000:0.0:1789760772.745579:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503735040 00000100:00100000:0.0:1789760772.745589:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503735040 00000100:00100000:0.0:1789760772.745656:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702838741330 time 702847415628 state 0x0 ref 1 00000100:00100000:0.0:1789760772.745681:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702847676044 time 702847415628 state 0x1 ref 1 00000100:00100000:0.0:1789760772.745711:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702847676044 time 702870317465 state 0x0 ref 1 00000100:00100000:0.0:1789760772.745723:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503735040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.745769:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75116544, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d30 00010000:00010000:0.0:1789760772.745779:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.745793:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760772.745797:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.745804:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.745816:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.750384:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985646 is committed 00002000:00010000:0.0:1789760772.750507:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75116544 -> 75120640 00002000:00010000:0.0:1789760772.750512:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146736 -> 146744 00010000:00010000:0.0:1789760772.750524:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.750536:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.750551:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.750566:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.750570:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.750580:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.750603:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.750610:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efd7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75116544->75120639] (req 75116544->75120639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.750702:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503735040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4977us (5399us total) trans 4294985646 rc 0/0 00000100:00100000:0.0:1789760772.750712:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18350 00000100:00100000:0.0:1789760772.750722:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702847676044 time 702870317465 state 0x2 ref 0 00000100:00100000:0.0:1789760772.753968:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.754050:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503735168 00000100:00100000:1.0:1789760772.754059:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503735168 00000100:00100000:1.0:1789760772.754114:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702847676044 time 702870317465 state 0x0 ref 1 00000100:00100000:1.0:1789760772.754129:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702870577881 time 702870317465 state 0x1 ref 1 00000100:00100000:1.0:1789760772.754158:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702870577881 time 702878764911 state 0x0 ref 1 00000100:00100000:1.0:1789760772.754170:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503735168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.754218:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75120640, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d38 00010000:00010000:1.0:1789760772.754226:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.754241:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.754246:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.754252:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.754265:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.763937:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985647 is committed 00002000:00010000:1.0:1789760772.764108:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75120640 -> 75124736 00002000:00010000:1.0:1789760772.764113:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146744 -> 146752 00010000:00010000:1.0:1789760772.764126:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.764139:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.764149:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.764156:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.764162:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.764170:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.764199:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.764206:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75120640->75124735] (req 75120640->75124735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.764306:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503735168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10132us (10342us total) trans 4294985647 rc 0/0 00000100:00100000:1.0:1789760772.764318:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18351 00000100:00100000:1.0:1789760772.764329:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702870577881 time 702878764911 state 0x2 ref 0 00000100:00100000:1.0:1789760772.774070:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503735296 00000100:00100000:1.0:1789760772.774081:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503735296 00000100:00100000:1.0:1789760772.774137:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702870577881 time 702878764911 state 0x0 ref 1 00000100:00100000:1.0:1789760772.774152:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702879025327 time 702878764911 state 0x1 ref 1 00000100:00100000:1.0:1789760772.774182:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702879025327 time 702898788920 state 0x0 ref 1 00000100:00100000:1.0:1789760772.774195:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503735296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.774242:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75124736, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d40 00010000:00010000:1.0:1789760772.774251:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.774267:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.774272:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.774278:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.774292:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.779329:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985648 is committed 00002000:00010000:1.0:1789760772.779445:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75124736 -> 75128832 00002000:00010000:1.0:1789760772.779450:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146752 -> 146760 00010000:00010000:1.0:1789760772.779463:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.779475:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.779483:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.779490:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.779494:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.779502:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.779521:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.779534:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efd8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75124736->75128831] (req 75124736->75128831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.779647:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503735296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5441us (5675us total) trans 4294985648 rc 0/0 00000100:00100000:1.0:1789760772.779658:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18352 00000100:00100000:1.0:1789760772.779668:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702879025327 time 702898788920 state 0x2 ref 0 00000100:00100000:1.0:1789760772.784376:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503735424 00000100:00100000:1.0:1789760772.784386:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503735424 00000100:00100000:1.0:1789760772.784479:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702879025327 time 702898788920 state 0x0 ref 1 00000100:00100000:1.0:1789760772.784494:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702899049336 time 702898788920 state 0x1 ref 1 00000100:00100000:1.0:1789760772.784517:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702899049336 time 702909123844 state 0x0 ref 1 00000100:00100000:1.0:1789760772.784528:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503735424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.784586:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75128832, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x23d48 00010000:00010000:1.0:1789760772.784595:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784611:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.784615:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.784621:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75128832->75132927] (req 75128832->75132927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.784649:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083efd93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75128832->75132927] (req 75128832->75132927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.791510:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985649 is committed 00000100:00100000:0.0:1789760772.798069:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.804219:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985650 is committed 00000100:00100000:0.0:1789760772.811935:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.812028:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503735680 00000100:00100000:1.0:1789760772.812038:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503735680 00000100:00100000:1.0:1789760772.812101:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702909384260 time 702922933520 state 0x0 ref 1 00000100:00100000:1.0:1789760772.812117:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702923193936 time 702922933520 state 0x1 ref 1 00000100:00100000:1.0:1789760772.812148:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702923193936 time 702936754133 state 0x0 ref 1 00000100:00100000:1.0:1789760772.812160:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503735680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.812207:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75137024, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23d58 00010000:00010000:1.0:1789760772.812216:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.812231:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.812235:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.812241:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.812254:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.817540:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985651 is committed 00002000:00010000:1.0:1789760772.818290:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75137024 -> 75141120 00002000:00010000:1.0:1789760772.818297:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146776 -> 146784 00010000:00010000:1.0:1789760772.818317:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.818333:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.818342:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.818352:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.818357:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.818377:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.818484:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.818493:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efda1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75137024->75141119] (req 75137024->75141119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.818711:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503735680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6548us (6791us total) trans 4294985651 rc 0/0 00000100:00100000:1.0:1789760772.818730:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18355 00000100:00100000:1.0:1789760772.818739:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702923193936 time 702936754133 state 0x2 ref 0 00000100:00100000:0.0:1789760772.823516:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.823693:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503735808 00000100:00100000:1.0:1789760772.823704:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503735808 00000100:00100000:1.0:1789760772.823770:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702923193936 time 702936754133 state 0x0 ref 1 00000100:00100000:1.0:1789760772.823785:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702937014549 time 702936754133 state 0x1 ref 1 00000100:00100000:1.0:1789760772.823817:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702937014549 time 702948423691 state 0x0 ref 1 00000100:00100000:1.0:1789760772.823829:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503735808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.823877:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75141120, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23d60 00010000:00010000:1.0:1789760772.823886:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.823901:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.823905:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.823912:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.824097:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.828901:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985652 is committed 00002000:00010000:1.0:1789760772.835139:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75141120 -> 75145216 00002000:00010000:1.0:1789760772.835148:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146784 -> 146792 00010000:00010000:1.0:1789760772.835163:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.835176:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.835185:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.835192:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.835197:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.835206:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.835224:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.835232:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efda8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75141120->75145215] (req 75141120->75145215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.835318:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503735808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11486us (11807us total) trans 4294985652 rc 0/0 00000100:00100000:1.0:1789760772.835328:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18356 00000100:00100000:1.0:1789760772.835338:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702937014549 time 702948423691 state 0x2 ref 0 00000100:00100000:0.0:1789760772.840611:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.846812:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985653 is committed 00000100:00100000:0.0:1789760772.853362:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.860489:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985654 is committed 00000100:00100000:0.0:1789760772.867600:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.872074:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503736192 00000100:00100000:1.0:1789760772.872089:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503736192 00000100:00100000:1.0:1789760772.872150:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702965686899 time 702978793847 state 0x0 ref 1 00000100:00100000:1.0:1789760772.872166:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702979054263 time 702978793847 state 0x1 ref 1 00000100:00100000:1.0:1789760772.872195:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702979054263 time 702996802458 state 0x0 ref 1 00000100:00100000:1.0:1789760772.872207:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503736192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.872249:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75153408, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23d78 00010000:00010000:1.0:1789760772.872258:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.872273:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760772.872277:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.872283:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.872296:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.879339:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985655 is committed 00002000:00010000:0.0:1789760772.880746:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75153408 -> 75157504 00002000:00010000:0.0:1789760772.880756:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146808 -> 146816 00010000:00010000:0.0:1789760772.880771:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.880784:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.880791:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.880798:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.880803:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.880812:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.880832:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.880839:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083efdbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75153408->75157503] (req 75153408->75157503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.880943:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503736192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8733us (13350us total) trans 4294985655 rc 0/0 00000100:00100000:0.0:1789760772.880955:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18359 00000100:00100000:0.0:1789760772.880964:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702979054263 time 702996802458 state 0x2 ref 0 00000100:00100000:0.0:1789760772.885141:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.891508:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503736320 00000100:00100000:1.0:1789760772.891524:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503736320 00000100:00100000:1.0:1789760772.891590:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702979054263 time 702996802458 state 0x0 ref 1 00000100:00100000:1.0:1789760772.891606:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702997062874 time 702996802458 state 0x1 ref 1 00000100:00100000:1.0:1789760772.891849:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702997062874 time 703016453646 state 0x0 ref 1 00000100:00100000:1.0:1789760772.891864:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.891931:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75157504, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23d80 00010000:00010000:1.0:1789760772.891940:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.891957:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760772.891962:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.891969:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.891990:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.897322:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985656 is committed 00002000:00010000:0.0:1789760772.897712:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75157504 -> 75161600 00002000:00010000:0.0:1789760772.897718:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146816 -> 146824 00010000:00010000:0.0:1789760772.897732:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.897745:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.897755:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.897762:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.897767:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.897777:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.897797:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.897805:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083efdc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75157504->75161599] (req 75157504->75161599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.897915:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6050us (12780us total) trans 4294985656 rc 0/0 00000100:00100000:0.0:1789760772.897930:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18360 00000100:00100000:0.0:1789760772.897941:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702997062874 time 703016453646 state 0x2 ref 0 00000100:00100000:1.0:1789760772.908825:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503736448 00000100:00100000:1.0:1789760772.908836:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503736448 00000100:00100000:1.0:1789760772.909031:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 702997062874 time 703016453646 state 0x0 ref 1 00000100:00100000:1.0:1789760772.909049:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703016714062 time 703016453646 state 0x1 ref 1 00000100:00100000:1.0:1789760772.909156:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703016714062 time 703033761829 state 0x0 ref 1 00000100:00100000:1.0:1789760772.909170:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.909225:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75161600, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23d88 00010000:00010000:1.0:1789760772.909234:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.909249:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760772.909254:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.909260:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.909274:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.918878:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75161600 -> 75165696 00002000:00010000:0.0:1789760772.918883:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146824 -> 146832 00010000:00010000:0.0:1789760772.918895:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.918908:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.918918:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.918992:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.919068:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.919079:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.919098:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.919111:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083efdcb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75161600->75165695] (req 75161600->75165695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.919217:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10045us (10709us total) trans 4294985657 rc 0/0 00000100:00100000:0.0:1789760772.919702:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18361 00000100:00100000:0.0:1789760772.919712:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703016714062 time 703033761829 state 0x2 ref 0 00000100:00100000:0.0:1789760772.926827:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503736576 00000100:00100000:0.0:1789760772.926836:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503736576 00000100:00100000:0.0:1789760772.926893:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703016714062 time 703033761829 state 0x0 ref 1 00000100:00100000:0.0:1789760772.926909:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703034022245 time 703033761829 state 0x1 ref 1 00000100:00100000:0.0:1789760772.927131:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703034022245 time 703051735844 state 0x0 ref 1 00000100:00100000:0.0:1789760772.927144:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503736576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.927192:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75165696, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23d90 00010000:00010000:0.0:1789760772.927201:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.927216:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760772.927222:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.927228:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.927247:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.932739:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75165696 -> 75169792 00002000:00010000:0.0:1789760772.932745:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146832 -> 146840 00010000:00010000:0.0:1789760772.932758:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.932770:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.932779:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.932786:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.932791:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.932800:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.932819:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.932826:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efdd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75165696->75169791] (req 75165696->75169791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.932907:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503736576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5761us (6350us total) trans 4294985658 rc 0/0 00000100:00100000:0.0:1789760772.932917:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18362 00000100:00100000:0.0:1789760772.932926:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703034022245 time 703051735844 state 0x2 ref 0 00000100:00100000:1.0:1789760772.935868:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760772.935957:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503736704 00000100:00100000:0.0:1789760772.935966:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503736704 00000100:00100000:0.0:1789760772.936015:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703034022245 time 703051735844 state 0x0 ref 1 00000100:00100000:0.0:1789760772.936027:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703051996260 time 703051735844 state 0x1 ref 1 00000100:00100000:0.0:1789760772.936080:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703051996260 time 703060687344 state 0x0 ref 1 00000100:00100000:0.0:1789760772.936089:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.936126:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75169792, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23d98 00010000:00010000:0.0:1789760772.936132:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.936147:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760772.936151:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.936157:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.936170:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.944104:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75169792 -> 75173888 00002000:00010000:0.0:1789760772.944109:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146840 -> 146848 00010000:00010000:0.0:1789760772.944121:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944134:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944143:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944150:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.944155:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944163:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944182:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.944189:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083efdd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75169792->75173887] (req 75169792->75173887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.944378:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8285us (8514us total) trans 4294985659 rc 0/0 00000100:00100000:0.0:1789760772.944437:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18363 00000100:00100000:0.0:1789760772.944447:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703051996260 time 703060687344 state 0x2 ref 0 00000100:00100000:1.0:1789760772.952496:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503736832 00000100:00100000:1.0:1789760772.952507:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503736832 00000100:00100000:1.0:1789760772.952562:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703051996260 time 703060687344 state 0x0 ref 1 00000100:00100000:1.0:1789760772.952577:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703060947760 time 703060687344 state 0x1 ref 1 00000100:00100000:1.0:1789760772.952610:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703060947760 time 703077215802 state 0x0 ref 1 00000100:00100000:1.0:1789760772.952622:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503736832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.952689:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75173888, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23da0 00010000:00010000:1.0:1789760772.952699:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.952719:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760772.952723:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.952729:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.952743:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.958306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985660 is committed 00002000:00010000:1.0:1789760772.959519:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75173888 -> 75177984 00002000:00010000:1.0:1789760772.959527:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146848 -> 146856 00010000:00010000:1.0:1789760772.959539:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.959553:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.959563:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.959571:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.959575:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.959584:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.959608:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.959615:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efde0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75173888->75177983] (req 75173888->75177983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.959818:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503736832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7193us (7467us total) trans 4294985660 rc 0/0 00000100:00100000:1.0:1789760772.959828:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18364 00000100:00100000:1.0:1789760772.959837:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703060947760 time 703077215802 state 0x2 ref 0 00000100:00100000:0.0:1789760772.962991:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503736960 00000100:00100000:0.0:1789760772.963001:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503736960 00000100:00100000:0.0:1789760772.963054:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703060947760 time 703077215802 state 0x0 ref 1 00000100:00100000:0.0:1789760772.963068:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703077476218 time 703077215802 state 0x1 ref 1 00000100:00100000:0.0:1789760772.963497:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703077476218 time 703088100966 state 0x0 ref 1 00000100:00100000:0.0:1789760772.963514:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760772.963564:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75177984, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23da8 00010000:00010000:0.0:1789760772.963573:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.963589:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760772.963593:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.963600:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.963613:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.969872:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985661 is committed 00002000:00010000:0.0:1789760772.970260:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75177984 -> 75182080 00002000:00010000:0.0:1789760772.970265:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146856 -> 146864 00010000:00010000:0.0:1789760772.970348:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.970362:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.970371:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.970378:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760772.970383:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.970429:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.970449:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.970458:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efde7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75177984->75182079] (req 75177984->75182079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.970841:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503736960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7033us (7706us total) trans 4294985661 rc 0/0 00000100:00100000:0.0:1789760772.970851:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18365 00000100:00100000:0.0:1789760772.970861:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703077476218 time 703088100966 state 0x2 ref 0 00000100:00100000:0.0:1789760772.975655:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760772.981050:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985662 is committed 00000100:00100000:1.0:1789760772.987330:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503737216 00000100:00100000:1.0:1789760772.987340:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503737216 00000100:00100000:1.0:1789760772.987432:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703088361382 time 703100557122 state 0x0 ref 1 00000100:00100000:1.0:1789760772.987447:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703100817538 time 703100557122 state 0x1 ref 1 00000100:00100000:1.0:1789760772.987468:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703100817538 time 703112074723 state 0x0 ref 1 00000100:00100000:1.0:1789760772.987480:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503737216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.987526:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75186176, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23db8 00010000:00010000:1.0:1789760772.987535:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.987551:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760772.987556:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.987562:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.987576:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.991667:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985663 is committed 00002000:00010000:1.0:1789760772.991839:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75186176 -> 75190272 00002000:00010000:1.0:1789760772.991845:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146872 -> 146880 00010000:00010000:1.0:1789760772.991858:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.991871:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.991880:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.991887:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760772.991892:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.991900:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.991919:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.991932:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efdf5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75186176->75190271] (req 75186176->75190271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.992018:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503737216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4536us (6342us total) trans 4294985663 rc 0/0 00000100:00100000:1.0:1789760772.992027:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18367 00000100:00100000:1.0:1789760772.992036:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703100817538 time 703112074723 state 0x2 ref 0 00000100:00100000:0.0:1789760772.995679:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760772.995749:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503737344 00000100:00100000:1.0:1789760772.995758:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503737344 00000100:00100000:1.0:1789760772.995808:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703100817538 time 703112074723 state 0x0 ref 1 00000100:00100000:1.0:1789760772.995822:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703112335139 time 703112074723 state 0x1 ref 1 00000100:00100000:1.0:1789760772.995855:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703112335139 time 703120462056 state 0x0 ref 1 00000100:00100000:1.0:1789760772.995868:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760772.995912:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75190272, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23dc0 00010000:00010000:1.0:1789760772.995921:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.995936:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760772.995941:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.995947:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.995965:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.005301:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985664 is committed 00002000:00010000:1.0:1789760773.005468:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75190272 -> 75194368 00002000:00010000:1.0:1789760773.005473:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146880 -> 146888 00010000:00010000:1.0:1789760773.005486:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.005499:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.005507:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.005514:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.005519:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.005527:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.005547:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.005555:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083efdfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75190272->75194367] (req 75190272->75194367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.005757:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9886us (10092us total) trans 4294985664 rc 0/0 00000100:00100000:1.0:1789760773.005767:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18368 00000100:00100000:1.0:1789760773.005777:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703112335139 time 703120462056 state 0x2 ref 0 00000100:00100000:0.0:1789760773.008732:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760773.019283:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985665 is committed 00000100:00100000:0.0:1789760773.024804:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503737600 00000100:00100000:0.0:1789760773.024813:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503737600 00000100:00100000:0.0:1789760773.024867:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703120722472 time 703133568917 state 0x0 ref 1 00000100:00100000:0.0:1789760773.024882:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703133829333 time 703133568917 state 0x1 ref 1 00000100:00100000:0.0:1789760773.024943:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703133829333 time 703149548002 state 0x0 ref 1 00000100:00100000:0.0:1789760773.024956:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.025004:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75198464, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23dd0 00010000:00010000:0.0:1789760773.025014:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.025029:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.025034:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.025041:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.025054:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.032033:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75198464 -> 75202560 00002000:00010000:1.0:1789760773.032038:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146896 -> 146904 00010000:00010000:1.0:1789760773.032052:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.032064:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.032073:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.032080:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.032086:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.032094:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.032115:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.032127:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75198464->75202559] (req 75198464->75202559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.032253:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7294us (7596us total) trans 4294985666 rc 0/0 00000100:00100000:1.0:1789760773.032264:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18370 00000100:00100000:1.0:1789760773.032274:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703133829333 time 703149548002 state 0x2 ref 0 00000100:00100000:0.0:1789760773.035232:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.035316:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503737728 00000100:00100000:1.0:1789760773.035326:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503737728 00000100:00100000:1.0:1789760773.035542:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703133829333 time 703149548002 state 0x0 ref 1 00000100:00100000:1.0:1789760773.035559:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703149808418 time 703149548002 state 0x1 ref 1 00000100:00100000:1.0:1789760773.035625:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703149808418 time 703160230844 state 0x0 ref 1 00000100:00100000:1.0:1789760773.035648:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503737728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.035758:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75202560, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23dd8 00010000:00010000:1.0:1789760773.035767:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.035783:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.035788:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.035794:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.035812:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.041337:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75202560 -> 75206656 00002000:00010000:1.0:1789760773.041344:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146904 -> 146912 00010000:00010000:1.0:1789760773.041372:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.041386:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.041415:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.041424:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.041429:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.041438:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.041458:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.041467:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083efe11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75202560->75206655] (req 75202560->75206655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.041565:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503737728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5915us (6337us total) trans 4294985667 rc 0/0 00000100:00100000:1.0:1789760773.041575:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18371 00000100:00100000:1.0:1789760773.041586:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703149808418 time 703160230844 state 0x2 ref 0 00000100:00100000:1.0:1789760773.044805:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.044894:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503737856 00000100:00100000:0.0:1789760773.044905:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503737856 00000100:00100000:0.0:1789760773.044961:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703149808418 time 703160230844 state 0x0 ref 1 00000100:00100000:0.0:1789760773.044976:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703160491260 time 703160230844 state 0x1 ref 1 00000100:00100000:0.0:1789760773.045005:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703160491260 time 703169612325 state 0x0 ref 1 00000100:00100000:0.0:1789760773.045018:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.045065:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75206656, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23de0 00010000:00010000:0.0:1789760773.045074:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.045090:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.045094:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.045100:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.045114:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.051090:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75206656 -> 75210752 00002000:00010000:0.0:1789760773.051095:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146912 -> 146920 00010000:00010000:0.0:1789760773.051108:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.051121:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.051130:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.051137:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.051141:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.051150:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.051169:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.051177:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75206656->75210751] (req 75206656->75210751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.051471:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6449us (6669us total) trans 4294985668 rc 0/0 00000100:00100000:0.0:1789760773.051483:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18372 00000100:00100000:0.0:1789760773.051492:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703160491260 time 703169612325 state 0x2 ref 0 00000100:00100000:0.0:1789760773.056231:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503737984 00000100:00100000:0.0:1789760773.056241:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503737984 00000100:00100000:0.0:1789760773.056295:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703160491260 time 703169612325 state 0x0 ref 1 00000100:00100000:0.0:1789760773.056311:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703169872741 time 703169612325 state 0x1 ref 1 00000100:00100000:0.0:1789760773.056341:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703169872741 time 703180948197 state 0x0 ref 1 00000100:00100000:0.0:1789760773.056354:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.056481:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75210752, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23de8 00010000:00010000:0.0:1789760773.056491:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.056507:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.056511:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.056518:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.056532:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.061901:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75210752 -> 75214848 00002000:00010000:0.0:1789760773.061906:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146920 -> 146928 00010000:00010000:0.0:1789760773.061919:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.061931:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.061947:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.061954:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.061959:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.061969:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.061987:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.061995:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75210752->75214847] (req 75210752->75214847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.062109:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503737984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5753us (5994us total) trans 4294985669 rc 0/0 00000100:00100000:0.0:1789760773.062117:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18373 00000100:00100000:0.0:1789760773.062124:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703169872741 time 703180948197 state 0x2 ref 0 00000100:00100000:1.0:1789760773.067377:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503738112 00000100:00100000:1.0:1789760773.067400:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503738112 00000100:00100000:1.0:1789760773.067461:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703169872741 time 703180948197 state 0x0 ref 1 00000100:00100000:1.0:1789760773.067477:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703181208613 time 703180948197 state 0x1 ref 1 00000100:00100000:1.0:1789760773.067511:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703181208613 time 703192117612 state 0x0 ref 1 00000100:00100000:1.0:1789760773.067524:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503738112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.067567:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75214848, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23df0 00010000:00010000:1.0:1789760773.067576:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.067591:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.067596:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.067602:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.067616:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.072596:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75214848 -> 75218944 00002000:00010000:1.0:1789760773.072602:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146928 -> 146936 00010000:00010000:1.0:1789760773.072616:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.072629:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.072648:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.072656:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.072661:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.072670:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.072869:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.072879:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75214848->75218943] (req 75214848->75218943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.073007:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503738112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5481us (6645us total) trans 4294985670 rc 0/0 00000100:00100000:1.0:1789760773.073019:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18374 00000100:00100000:1.0:1789760773.073029:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703181208613 time 703192117612 state 0x2 ref 0 00000100:00100000:1.0:1789760773.085859:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.085964:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503738240 00000100:00100000:0.0:1789760773.085975:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503738240 00000100:00100000:0.0:1789760773.086038:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703181208613 time 703192117612 state 0x0 ref 1 00000100:00100000:0.0:1789760773.086053:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703192378028 time 703192117612 state 0x1 ref 1 00000100:00100000:0.0:1789760773.086084:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703192378028 time 703210691262 state 0x0 ref 1 00000100:00100000:0.0:1789760773.086096:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503738240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.086141:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75218944, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23df8 00010000:00010000:0.0:1789760773.086150:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.086166:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.086171:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.086177:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.086201:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.091578:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75218944 -> 75223040 00002000:00010000:0.0:1789760773.091584:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146936 -> 146944 00010000:00010000:0.0:1789760773.091597:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.091608:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.091618:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.091639:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.091645:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.091654:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.091785:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.091794:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efe2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75218944->75223039] (req 75218944->75223039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.091927:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503738240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5828us (6074us total) trans 4294985671 rc 0/0 00000100:00100000:0.0:1789760773.091937:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18375 00000100:00100000:0.0:1789760773.091948:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703192378028 time 703210691262 state 0x2 ref 0 00000100:00100000:0.0:1789760773.106017:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503738368 00000100:00100000:0.0:1789760773.106030:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503738368 00000100:00100000:1.0:1789760773.106168:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703192378028 time 703210691262 state 0x0 ref 1 00000100:00100000:1.0:1789760773.106184:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703210951678 time 703210691262 state 0x1 ref 1 00000100:00100000:1.0:1789760773.106218:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703210951678 time 703230824688 state 0x0 ref 1 00000100:00100000:1.0:1789760773.106231:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503738368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.106277:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75223040, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e00 00010000:00010000:1.0:1789760773.106287:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.106303:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.106308:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.106314:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.106328:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.117920:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985672 is committed 00002000:00010000:1.0:1789760773.118056:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75223040 -> 75227136 00002000:00010000:1.0:1789760773.118061:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146944 -> 146952 00010000:00010000:1.0:1789760773.118083:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.118097:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.118107:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.118114:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.118119:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.118128:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.118147:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.118154:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75223040->75227135] (req 75223040->75227135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.118253:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503738368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12020us (13883us total) trans 4294985672 rc 0/0 00000100:00100000:1.0:1789760773.118264:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18376 00000100:00100000:1.0:1789760773.118274:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703210951678 time 703230824688 state 0x2 ref 0 00000100:00100000:3.0:1789760773.122574:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503738496 00000100:00100000:3.0:1789760773.122585:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503738496 00000100:00100000:3.0:1789760773.122647:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703210951678 time 703230824688 state 0x0 ref 1 00000100:00100000:3.0:1789760773.122661:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703231085104 time 703230824688 state 0x1 ref 1 00000100:00100000:3.0:1789760773.122701:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703231085104 time 703247308004 state 0x0 ref 1 00000100:00100000:3.0:1789760773.122714:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503738496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760773.122761:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75227136, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e08 00010000:00010000:3.0:1789760773.122769:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.122786:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760773.122790:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.122796:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.122809:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.133362:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985673 is committed 00002000:00010000:3.0:1789760773.136144:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75227136 -> 75231232 00002000:00010000:3.0:1789760773.136154:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146952 -> 146960 00010000:00010000:3.0:1789760773.136170:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.136183:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.136192:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.136204:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760773.136209:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.136218:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.136237:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.136244:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75227136->75231231] (req 75227136->75231231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.136427:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503738496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13678us (14029us total) trans 4294985673 rc 0/0 00000100:00100000:3.0:1789760773.136438:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18377 00000100:00100000:3.0:1789760773.136447:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703231085104 time 703247308004 state 0x2 ref 0 00000100:00100000:1.0:1789760773.141490:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503738624 00000100:00100000:1.0:1789760773.141502:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503738624 00000100:00100000:1.0:1789760773.141557:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703231085104 time 703247308004 state 0x0 ref 1 00000100:00100000:1.0:1789760773.141572:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703247568420 time 703247308004 state 0x1 ref 1 00000100:00100000:1.0:1789760773.141602:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703247568420 time 703266208735 state 0x0 ref 1 00000100:00100000:1.0:1789760773.141617:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503738624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.141674:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75231232, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e10 00010000:00010000:1.0:1789760773.141683:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.141707:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.141712:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.141717:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.141731:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.148759:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75231232 -> 75235328 00002000:00010000:1.0:1789760773.148764:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146960 -> 146968 00010000:00010000:1.0:1789760773.148777:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.148794:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.148803:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.148811:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.148815:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.148824:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.148845:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.148853:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75231232->75235327] (req 75231232->75235327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.149086:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503738624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7467us (8852us total) trans 4294985674 rc 0/0 00000100:00100000:1.0:1789760773.149098:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18378 00000100:00100000:1.0:1789760773.149107:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703247568420 time 703266208735 state 0x2 ref 0 00000100:00100000:1.0:1789760773.155375:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503738752 00000100:00100000:1.0:1789760773.155387:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503738752 00000100:00100000:1.0:1789760773.155556:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703247568420 time 703266208735 state 0x0 ref 1 00000100:00100000:1.0:1789760773.155571:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703266469151 time 703266208735 state 0x1 ref 1 00000100:00100000:1.0:1789760773.155605:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703266469151 time 703280207755 state 0x0 ref 1 00000100:00100000:1.0:1789760773.155617:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503738752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.155683:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75235328, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e18 00010000:00010000:1.0:1789760773.155692:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.155706:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.155711:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.155718:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.155730:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.170103:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75235328 -> 75239424 00002000:00010000:1.0:1789760773.170114:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146968 -> 146976 00010000:00010000:1.0:1789760773.170129:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.170141:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.170154:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.170161:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.170166:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.170174:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.170193:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.170201:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75235328->75239423] (req 75235328->75239423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.170309:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503738752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14689us (15407us total) trans 4294985675 rc 0/0 00000100:00100000:1.0:1789760773.170320:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18379 00000100:00100000:1.0:1789760773.170330:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703266469151 time 703280207755 state 0x2 ref 0 00000100:00100000:1.0:1789760773.176609:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503738880 00000100:00100000:1.0:1789760773.176621:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503738880 00000100:00100000:1.0:1789760773.176749:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703266469151 time 703280207755 state 0x0 ref 1 00000100:00100000:1.0:1789760773.176764:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703280468171 time 703280207755 state 0x1 ref 1 00000100:00100000:1.0:1789760773.176832:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703280468171 time 703301438233 state 0x0 ref 1 00000100:00100000:1.0:1789760773.176846:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503738880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.176899:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75239424, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e20 00010000:00010000:1.0:1789760773.176907:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.176921:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.176925:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.176931:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.176943:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.184993:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985676 is committed 00002000:00010000:1.0:1789760773.185123:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75239424 -> 75243520 00002000:00010000:1.0:1789760773.185128:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146976 -> 146984 00010000:00010000:1.0:1789760773.185141:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.185154:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.185161:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.185173:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.185178:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.185187:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.185205:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.185213:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75239424->75243519] (req 75239424->75243519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.185302:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503738880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8453us (8874us total) trans 4294985676 rc 0/0 00000100:00100000:1.0:1789760773.185310:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18380 00000100:00100000:1.0:1789760773.185320:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703280468171 time 703301438233 state 0x2 ref 0 00000100:00100000:3.0:1789760773.196423:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503739008 00000100:00100000:3.0:1789760773.196434:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503739008 00000100:00100000:3.0:1789760773.196489:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703280468171 time 703301438233 state 0x0 ref 1 00000100:00100000:3.0:1789760773.196504:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703301698649 time 703301438233 state 0x1 ref 1 00000100:00100000:3.0:1789760773.196539:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703301698649 time 703321145071 state 0x0 ref 1 00000100:00100000:3.0:1789760773.196552:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503739008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760773.196610:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75243520, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e28 00010000:00010000:3.0:1789760773.196619:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.196646:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760773.196651:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.196658:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.196673:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760773.207003:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75243520 -> 75247616 00002000:00010000:3.0:1789760773.207008:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146984 -> 146992 00010000:00010000:3.0:1789760773.207022:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.207034:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.207044:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.207051:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760773.207057:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.207066:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.207086:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.207094:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efe57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75243520->75247615] (req 75243520->75247615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.207185:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503739008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10631us (11004us total) trans 4294985677 rc 0/0 00000100:00100000:3.0:1789760773.207196:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18381 00000100:00100000:3.0:1789760773.207205:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703301698649 time 703321145071 state 0x2 ref 0 00000100:00100000:1.0:1789760773.213618:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503739136 00000100:00100000:1.0:1789760773.213638:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503739136 00000100:00100000:1.0:1789760773.213695:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703301698649 time 703321145071 state 0x0 ref 1 00000100:00100000:1.0:1789760773.213710:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703321405487 time 703321145071 state 0x1 ref 1 00000100:00100000:1.0:1789760773.213741:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703321405487 time 703338347962 state 0x0 ref 1 00000100:00100000:1.0:1789760773.213757:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503739136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.213806:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75247616, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e30 00010000:00010000:1.0:1789760773.213816:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.213832:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.213837:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.213843:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.213859:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.222007:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75247616 -> 75251712 00002000:00010000:0.0:1789760773.222014:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 146992 -> 147000 00010000:00010000:0.0:1789760773.222028:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.222048:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.222057:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.222065:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.222069:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.222079:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.222099:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.222108:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083efe5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75247616->75251711] (req 75247616->75251711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.222521:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503739136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8758us (9649us total) trans 4294985678 rc 0/0 00000100:00100000:0.0:1789760773.222532:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18382 00000100:00100000:0.0:1789760773.222542:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703321405487 time 703338347962 state 0x2 ref 0 00000100:00100000:3.0:1789760773.230945:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.232603:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503739264 00000100:00100000:1.0:1789760773.232617:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503739264 00000100:00100000:1.0:1789760773.232753:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703321405487 time 703338347962 state 0x0 ref 1 00000100:00100000:1.0:1789760773.232770:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703338608378 time 703338347962 state 0x1 ref 1 00000100:00100000:1.0:1789760773.232837:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703338608378 time 703357443488 state 0x0 ref 1 00000100:00100000:1.0:1789760773.232852:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503739264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.232900:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75251712, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e38 00010000:00010000:1.0:1789760773.232910:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.232931:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.232936:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.232945:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.232957:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.239525:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75251712 -> 75255808 00002000:00010000:1.0:1789760773.239531:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147000 -> 147008 00010000:00010000:1.0:1789760773.239544:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.239556:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.239566:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.239573:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.239578:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.239587:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.239612:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.239619:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75251712->75255807] (req 75251712->75255807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.239725:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503739264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6872us (8786us total) trans 4294985679 rc 0/0 00000100:00100000:1.0:1789760773.239734:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18383 00000100:00100000:1.0:1789760773.239743:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703338608378 time 703357443488 state 0x2 ref 0 00000100:00100000:3.0:1789760773.245542:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760773.253447:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985680 is committed 00000100:00100000:3.0:1789760773.259362:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503739520 00000100:00100000:3.0:1789760773.259376:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503739520 00000100:00100000:0.0:1789760773.259656:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703357703904 time 703371900690 state 0x0 ref 1 00000100:00100000:0.0:1789760773.259673:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703372161106 time 703371900690 state 0x1 ref 1 00000100:00100000:0.0:1789760773.259712:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703372161106 time 703384318792 state 0x0 ref 1 00000100:00100000:0.0:1789760773.259726:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503739520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.259771:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75259904, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e48 00010000:00010000:0.0:1789760773.259781:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.259795:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.259799:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.259806:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.259819:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.264584:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985681 is committed 00002000:00010000:0.0:1789760773.265004:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75259904 -> 75264000 00002000:00010000:0.0:1789760773.265010:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147016 -> 147024 00010000:00010000:0.0:1789760773.265024:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.265037:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.265047:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.265056:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.265061:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.265068:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.265087:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.265095:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083efe73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75259904->75263999] (req 75259904->75263999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.265227:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503739520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5499us (8015us total) trans 4294985681 rc 0/0 00000100:00100000:0.0:1789760773.265237:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18385 00000100:00100000:0.0:1789760773.265248:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703372161106 time 703384318792 state 0x2 ref 0 00000100:00100000:0.0:1789760773.276123:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760773.281071:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985682 is committed 00000100:00100000:3.0:1789760773.285807:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.285967:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503739776 00000100:00100000:1.0:1789760773.285978:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503739776 00000100:00100000:1.0:1789760773.286031:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703384579208 time 703401258957 state 0x0 ref 1 00000100:00100000:1.0:1789760773.286046:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703401519373 time 703401258957 state 0x1 ref 1 00000100:00100000:1.0:1789760773.286077:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703401519373 time 703410683962 state 0x0 ref 1 00000100:00100000:1.0:1789760773.286089:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503739776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.286135:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75268096, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e58 00010000:00010000:1.0:1789760773.286144:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.286158:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.286163:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.286170:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.286183:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.290515:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985683 is committed 00002000:00010000:1.0:1789760773.290775:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75268096 -> 75272192 00002000:00010000:1.0:1789760773.290780:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147032 -> 147040 00010000:00010000:1.0:1789760773.290794:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.290806:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.290814:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.290820:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.290825:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.290833:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.290853:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.290864:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75268096->75272191] (req 75268096->75272191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.290962:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503739776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4870us (5160us total) trans 4294985683 rc 0/0 00000100:00100000:1.0:1789760773.290972:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18387 00000100:00100000:1.0:1789760773.290981:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703401519373 time 703410683962 state 0x2 ref 0 00000100:00100000:0.0:1789760773.294308:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.294415:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503739904 00000100:00100000:1.0:1789760773.294425:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503739904 00000100:00100000:1.0:1789760773.294476:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703401519373 time 703410683962 state 0x0 ref 1 00000100:00100000:1.0:1789760773.294493:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703410944378 time 703410683962 state 0x1 ref 1 00000100:00100000:1.0:1789760773.294531:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703410944378 time 703419130404 state 0x0 ref 1 00000100:00100000:1.0:1789760773.294550:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503739904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.294598:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75272192, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e60 00010000:00010000:1.0:1789760773.294608:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.294624:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.294638:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.294645:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.294664:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.301455:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75272192 -> 75276288 00002000:00010000:1.0:1789760773.301461:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147040 -> 147048 00010000:00010000:1.0:1789760773.301476:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.301490:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.301500:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.301508:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.301513:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.301522:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.301542:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.301562:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75272192->75276287] (req 75272192->75276287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.301746:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503739904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7197us (7441us total) trans 4294985684 rc 0/0 00000100:00100000:1.0:1789760773.301757:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18388 00000100:00100000:1.0:1789760773.301767:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703410944378 time 703419130404 state 0x2 ref 0 00000100:00100000:0.0:1789760773.305824:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760773.312918:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75276288 -> 75280384 00002000:00010000:0.0:1789760773.312924:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147048 -> 147056 00010000:00010000:0.0:1789760773.312936:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efe8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75276288->75280383] (req 75276288->75280383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.312954:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efe8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75276288->75280383] (req 75276288->75280383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.312962:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efe8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75276288->75280383] (req 75276288->75280383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.312970:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.312975:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efe8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75276288->75280383] (req 75276288->75280383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.313159:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efe8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75276288->75280383] (req 75276288->75280383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.313181:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efe8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75276288->75280383] (req 75276288->75280383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.313189:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083efe8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75276288->75280383] (req 75276288->75280383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.313344:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503740032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7221us (7537us total) trans 4294985685 rc 0/0 00000100:00100000:0.0:1789760773.313362:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18389 00000100:00100000:0.0:1789760773.313372:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703419390820 time 703430714122 state 0x2 ref 0 00000100:00100000:3.0:1789760773.319028:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.319426:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703430974538 time 703443991985 state 0x0 ref 1 00000100:00100000:1.0:1789760773.319444:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503740160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.319495:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75280384, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e70 00010000:00010000:1.0:1789760773.319504:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.319520:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.319525:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.319532:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.319546:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.324232:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75280384 -> 75284480 00002000:00010000:1.0:1789760773.324238:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147056 -> 147064 00010000:00010000:1.0:1789760773.324251:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.324271:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.324280:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.324288:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.324293:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.324302:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.324320:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.324327:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efe96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75280384->75284479] (req 75280384->75284479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.324458:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503740160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5013us (5434us total) trans 4294985686 rc 0/0 00000100:00100000:1.0:1789760773.324469:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18390 00000100:00100000:1.0:1789760773.324479:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703430974538 time 703443991985 state 0x2 ref 0 00000100:00100000:1.0:1789760773.327164:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503740288 00000100:00100000:1.0:1789760773.327175:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503740288 00000100:00100000:1.0:1789760773.327225:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703430974538 time 703443991985 state 0x0 ref 1 00000100:00100000:1.0:1789760773.327239:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703444252401 time 703443991985 state 0x1 ref 1 00000100:00100000:1.0:1789760773.327268:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703444252401 time 703451875578 state 0x0 ref 1 00000100:00100000:1.0:1789760773.327280:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503740288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.327325:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75284480, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e78 00010000:00010000:1.0:1789760773.327334:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.327350:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.327354:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.327361:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.327374:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.335507:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75284480 -> 75288576 00002000:00010000:1.0:1789760773.335513:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147064 -> 147072 00010000:00010000:1.0:1789760773.335527:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.335539:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.335554:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.335560:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.335565:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.335575:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.335717:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.335727:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efe9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75284480->75288575] (req 75284480->75288575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.335848:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503740288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8564us (8870us total) trans 4294985687 rc 0/0 00000100:00100000:1.0:1789760773.335859:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18391 00000100:00100000:1.0:1789760773.335868:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703444252401 time 703451875578 state 0x2 ref 0 00000100:00100000:0.0:1789760773.342084:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503740416 00000100:00100000:0.0:1789760773.342095:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503740416 00000100:00100000:0.0:1789760773.342148:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703444252401 time 703451875578 state 0x0 ref 1 00000100:00100000:0.0:1789760773.342163:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703452135994 time 703451875578 state 0x1 ref 1 00000100:00100000:0.0:1789760773.342195:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703452135994 time 703466801517 state 0x0 ref 1 00000100:00100000:0.0:1789760773.342207:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503740416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.342251:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75288576, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e80 00010000:00010000:0.0:1789760773.342259:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.342273:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.342277:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.342283:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.342296:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.350334:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75288576 -> 75292672 00002000:00010000:0.0:1789760773.350340:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147072 -> 147080 00010000:00010000:0.0:1789760773.350494:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.350508:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.350587:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.350596:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.350602:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.350610:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.350808:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.350818:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efea4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75288576->75292671] (req 75288576->75292671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.350911:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503740416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8700us (9101us total) trans 4294985688 rc 0/0 00000100:00100000:0.0:1789760773.350921:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18392 00000100:00100000:0.0:1789760773.350931:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703452135994 time 703466801517 state 0x2 ref 0 00000100:00100000:3.0:1789760773.355980:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.356085:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503740544 00000100:00100000:0.0:1789760773.356095:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503740544 00000100:00100000:0.0:1789760773.356272:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703452135994 time 703466801517 state 0x0 ref 1 00000100:00100000:0.0:1789760773.356289:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703467061933 time 703466801517 state 0x1 ref 1 00000100:00100000:0.0:1789760773.356327:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703467061933 time 703480934477 state 0x0 ref 1 00000100:00100000:0.0:1789760773.356339:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503740544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.356528:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75292672, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e88 00010000:00010000:0.0:1789760773.356537:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.356552:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.356556:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.356562:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.356575:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.369719:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75292672 -> 75296768 00002000:00010000:1.0:1789760773.369729:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147080 -> 147088 00010000:00010000:1.0:1789760773.369743:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.369858:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.369866:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.369873:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.369878:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.369887:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.369905:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.369911:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083efeab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75292672->75296767] (req 75292672->75296767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.370059:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503740544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13717us (14088us total) trans 4294985689 rc 0/0 00000100:00100000:1.0:1789760773.370069:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18393 00000100:00100000:1.0:1789760773.370079:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703467061933 time 703480934477 state 0x2 ref 0 00000100:00100000:1.0:1789760773.377501:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503740672 00000100:00100000:1.0:1789760773.377512:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503740672 00000100:00100000:1.0:1789760773.377570:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703467061933 time 703480934477 state 0x0 ref 1 00000100:00100000:1.0:1789760773.377583:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703481194893 time 703480934477 state 0x1 ref 1 00000100:00100000:1.0:1789760773.377602:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703481194893 time 703502209388 state 0x0 ref 1 00000100:00100000:1.0:1789760773.377615:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503740672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.377673:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75296768, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e90 00010000:00010000:1.0:1789760773.377682:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.377698:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.377703:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.377709:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.377722:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.385539:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75296768 -> 75300864 00002000:00010000:1.0:1789760773.385545:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147088 -> 147096 00010000:00010000:1.0:1789760773.385558:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.385570:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.385585:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.385592:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.385597:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.385606:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.385642:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.385650:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75296768->75300863] (req 75296768->75300863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.385741:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503740672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8123us (8784us total) trans 4294985690 rc 0/0 00000100:00100000:1.0:1789760773.385753:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18394 00000100:00100000:1.0:1789760773.385763:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703481194893 time 703502209388 state 0x2 ref 0 00000100:00100000:1.0:1789760773.394676:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503740800 00000100:00100000:1.0:1789760773.394687:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503740800 00000100:00100000:1.0:1789760773.394745:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703481194893 time 703502209388 state 0x0 ref 1 00000100:00100000:1.0:1789760773.394761:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703502469804 time 703502209388 state 0x1 ref 1 00000100:00100000:1.0:1789760773.394795:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703502469804 time 703519401495 state 0x0 ref 1 00000100:00100000:1.0:1789760773.394808:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503740800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.394853:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75300864, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23e98 00010000:00010000:1.0:1789760773.394862:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.394878:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.394883:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.394889:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.394903:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.401507:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985691 is committed 00002000:00010000:1.0:1789760773.401605:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75300864 -> 75304960 00002000:00010000:1.0:1789760773.401610:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147096 -> 147104 00010000:00010000:1.0:1789760773.401640:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.401652:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.401661:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.401857:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.401864:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.401874:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.401895:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.401903:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083efeb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75300864->75304959] (req 75300864->75304959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.402007:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503740800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7196us (7517us total) trans 4294985691 rc 0/0 00000100:00100000:1.0:1789760773.402017:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18395 00000100:00100000:1.0:1789760773.402028:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703502469804 time 703519401495 state 0x2 ref 0 00000100:00100000:3.0:1789760773.410306:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.413121:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503740928 00000100:00100000:0.0:1789760773.413137:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503740928 00000100:00100000:0.0:1789760773.413204:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703502469804 time 703519401495 state 0x0 ref 1 00000100:00100000:0.0:1789760773.413220:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703519661911 time 703519401495 state 0x1 ref 1 00000100:00100000:0.0:1789760773.413304:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703519661911 time 703537909800 state 0x0 ref 1 00000100:00100000:0.0:1789760773.413318:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503740928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.413375:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75304960, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23ea0 00010000:00010000:0.0:1789760773.413384:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efec0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.413424:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.413428:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.413435:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75304960->75309055] (req 75304960->75309055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.413449:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083efec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75304960->75309055] (req 75304960->75309055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.418092:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985692 is committed 00000100:00100000:3.0:1789760773.421518:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760773.427209:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985693 is committed 00002000:00010000:3.0:1789760773.427313:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75309056 -> 75313152 00002000:00010000:3.0:1789760773.427319:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147112 -> 147120 00010000:00010000:3.0:1789760773.427332:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083efec7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75309056->75313151] (req 75309056->75313151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.427345:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083efec7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75309056->75313151] (req 75309056->75313151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.427776:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083efec7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75309056->75313151] (req 75309056->75313151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.427785:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760773.427790:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083efec7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75309056->75313151] (req 75309056->75313151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.427799:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083efec7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75309056->75313151] (req 75309056->75313151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.427818:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083efec7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75309056->75313151] (req 75309056->75313151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.427826:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083efec7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75309056->75313151] (req 75309056->75313151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.427918:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503741056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6167us (6405us total) trans 4294985693 rc 0/0 00000100:00100000:3.0:1789760773.427928:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18397 00000100:00100000:3.0:1789760773.427938:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703538170216 time 703546341360 state 0x2 ref 0 00000100:00100000:0.0:1789760773.434625:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.434914:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503741184 00000100:00100000:1.0:1789760773.434924:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503741184 00000100:00100000:1.0:1789760773.434980:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703538170216 time 703546341360 state 0x0 ref 1 00000100:00100000:1.0:1789760773.435005:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703546601776 time 703546341360 state 0x1 ref 1 00000100:00100000:1.0:1789760773.435033:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703546601776 time 703559640205 state 0x0 ref 1 00000100:00100000:1.0:1789760773.435048:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503741184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.435102:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75313152, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23eb0 00010000:00010000:1.0:1789760773.435111:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.435126:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.435130:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.435137:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.435150:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.444758:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985694 is committed 00002000:00010000:1.0:1789760773.447660:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75313152 -> 75317248 00002000:00010000:1.0:1789760773.447679:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147120 -> 147128 00010000:00010000:1.0:1789760773.447693:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.447707:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.447716:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.447723:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.447729:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.447738:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.447758:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.447766:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083efece lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75313152->75317247] (req 75313152->75317247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.447877:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503741184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12825us (13256us total) trans 4294985694 rc 0/0 00000100:00100000:1.0:1789760773.447887:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18398 00000100:00100000:1.0:1789760773.447896:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703546601776 time 703559640205 state 0x2 ref 0 00000100:00100000:1.0:1789760773.453277:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.453368:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503741312 00000100:00100000:0.0:1789760773.453378:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503741312 00000100:00100000:0.0:1789760773.453480:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703546601776 time 703559640205 state 0x0 ref 1 00000100:00100000:0.0:1789760773.453496:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703559900621 time 703559640205 state 0x1 ref 1 00000100:00100000:0.0:1789760773.453527:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703559900621 time 703578133355 state 0x0 ref 1 00000100:00100000:0.0:1789760773.453540:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503741312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.453586:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75317248, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23eb8 00010000:00010000:0.0:1789760773.453596:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.453611:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.453616:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.453622:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.453644:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.460773:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985695 is committed 00002000:00010000:1.0:1789760773.460947:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75317248 -> 75321344 00002000:00010000:1.0:1789760773.460953:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147128 -> 147136 00010000:00010000:1.0:1789760773.460967:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.460979:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.460988:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.460996:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.461001:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.461011:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.461031:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.461038:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083efed5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75317248->75321343] (req 75317248->75321343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.461325:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503741312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7782us (8196us total) trans 4294985695 rc 0/0 00000100:00100000:1.0:1789760773.461336:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18399 00000100:00100000:1.0:1789760773.461345:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703559900621 time 703578133355 state 0x2 ref 0 00000100:00100000:3.0:1789760773.475773:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760773.481377:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985696 is committed 00000100:00100000:0.0:1789760773.503439:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760773.503525:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503741568 00000100:00100000:3.0:1789760773.503536:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503741568 00000100:00100000:3.0:1789760773.503590:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703578393771 time 703600764931 state 0x0 ref 1 00000100:00100000:3.0:1789760773.503607:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703601025347 time 703600764931 state 0x1 ref 1 00000100:00100000:3.0:1789760773.503637:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703601025347 time 703628235086 state 0x0 ref 1 00000100:00100000:3.0:1789760773.503651:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503741568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760773.503696:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75325440, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23ec8 00010000:00010000:3.0:1789760773.503706:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503722:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760773.503726:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.503733:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.503747:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.511035:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985697 is committed 00002000:00010000:3.0:1789760773.511216:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75325440 -> 75329536 00002000:00010000:3.0:1789760773.511220:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147144 -> 147152 00010000:00010000:3.0:1789760773.511231:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.511240:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.511245:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.511250:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760773.511253:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.511260:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.511274:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.511278:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083efee3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75325440->75329535] (req 75325440->75329535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.511358:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503741568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7702us (7921us total) trans 4294985697 rc 0/0 00000100:00100000:3.0:1789760773.511366:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18401 00000100:00100000:3.0:1789760773.511373:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703601025347 time 703628235086 state 0x2 ref 0 00000100:00100000:1.0:1789760773.517351:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760773.528423:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985698 is committed 00000100:00100000:0.0:1789760773.539111:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.539362:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503741824 00000100:00100000:1.0:1789760773.539373:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503741824 00000100:00100000:1.0:1789760773.539462:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703628495502 time 703642828467 state 0x0 ref 1 00000100:00100000:1.0:1789760773.539486:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703643088883 time 703642828467 state 0x1 ref 1 00000100:00100000:1.0:1789760773.539517:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703643088883 time 703664123698 state 0x0 ref 1 00000100:00100000:1.0:1789760773.539530:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503741824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.539578:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75333632, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23ed8 00010000:00010000:1.0:1789760773.539586:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.539602:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.539606:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.539612:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.539625:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.548311:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985699 is committed 00002000:00010000:1.0:1789760773.548483:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75333632 -> 75337728 00002000:00010000:1.0:1789760773.548495:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147160 -> 147168 00010000:00010000:1.0:1789760773.548508:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.548522:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.548531:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.548547:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.548552:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.548561:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.548579:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.548586:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efef1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75333632->75337727] (req 75333632->75337727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.548729:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503741824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9195us (9621us total) trans 4294985699 rc 0/0 00000100:00100000:1.0:1789760773.548740:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18403 00000100:00100000:1.0:1789760773.548749:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703643088883 time 703664123698 state 0x2 ref 0 00000100:00100000:0.0:1789760773.554535:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760773.562781:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985700 is committed 00000100:00100000:3.0:1789760773.568531:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.568841:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503742080 00000100:00100000:1.0:1789760773.568851:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503742080 00000100:00100000:1.0:1789760773.568916:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703664384114 time 703679391070 state 0x0 ref 1 00000100:00100000:1.0:1789760773.568940:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703679651486 time 703679391070 state 0x1 ref 1 00000100:00100000:1.0:1789760773.569010:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703679651486 time 703693615688 state 0x0 ref 1 00000100:00100000:1.0:1789760773.569026:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503742080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.569075:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75341824, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23ee8 00010000:00010000:1.0:1789760773.569086:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.569102:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.569107:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.569113:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.569139:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.576481:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985701 is committed 00002000:00010000:1.0:1789760773.577058:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75341824 -> 75345920 00002000:00010000:1.0:1789760773.577064:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147176 -> 147184 00010000:00010000:1.0:1789760773.577078:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.577091:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.577099:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.577107:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.577113:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.577123:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.577468:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.577476:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efeff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75341824->75345919] (req 75341824->75345919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.577561:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503742080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8532us (9199us total) trans 4294985701 rc 0/0 00000100:00100000:1.0:1789760773.577572:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18405 00000100:00100000:1.0:1789760773.577582:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703679651486 time 703693615688 state 0x2 ref 0 00000100:00100000:0.0:1789760773.583006:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.583112:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503742208 00000100:00100000:1.0:1789760773.583122:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503742208 00000100:00100000:1.0:1789760773.583236:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703679651486 time 703693615688 state 0x0 ref 1 00000100:00100000:1.0:1789760773.583251:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703693876104 time 703693615688 state 0x1 ref 1 00000100:00100000:1.0:1789760773.583317:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703693876104 time 703707922484 state 0x0 ref 1 00000100:00100000:1.0:1789760773.583329:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503742208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.583386:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75345920, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23ef0 00010000:00010000:1.0:1789760773.583422:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.583437:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.583441:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.583447:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.583461:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.590855:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985702 is committed 00002000:00010000:1.0:1789760773.591122:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75345920 -> 75350016 00002000:00010000:1.0:1789760773.591137:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147184 -> 147192 00010000:00010000:1.0:1789760773.591150:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.591164:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.591173:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.591180:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.591185:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.591194:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.591213:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.591221:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75345920->75350015] (req 75345920->75350015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.591311:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503742208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7979us (8381us total) trans 4294985702 rc 0/0 00000100:00100000:1.0:1789760773.591321:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18406 00000100:00100000:1.0:1789760773.591331:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703693876104 time 703707922484 state 0x2 ref 0 00000100:00100000:0.0:1789760773.598438:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.598582:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503742336 00000100:00100000:1.0:1789760773.598591:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503742336 00000100:00100000:1.0:1789760773.598672:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703693876104 time 703707922484 state 0x0 ref 1 00000100:00100000:1.0:1789760773.598697:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703708182900 time 703707922484 state 0x1 ref 1 00000100:00100000:1.0:1789760773.598727:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703708182900 time 703723333676 state 0x0 ref 1 00000100:00100000:1.0:1789760773.598741:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503742336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.598794:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75350016, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23ef8 00010000:00010000:1.0:1789760773.598804:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.598820:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.598826:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.598832:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.598845:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.610550:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985703 is committed 00002000:00010000:1.0:1789760773.610985:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75350016 -> 75354112 00002000:00010000:1.0:1789760773.610991:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147192 -> 147200 00010000:00010000:1.0:1789760773.611005:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.611017:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.611025:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.611033:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.611037:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.611046:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.611067:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.611079:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75350016->75354111] (req 75350016->75354111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.611230:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503742336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12488us (12797us total) trans 4294985703 rc 0/0 00000100:00100000:1.0:1789760773.611241:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18407 00000100:00100000:1.0:1789760773.611250:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703708182900 time 703723333676 state 0x2 ref 0 00000100:00100000:3.0:1789760773.616552:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760773.628776:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985704 is committed 00000100:00100000:0.0:1789760773.635031:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760773.635307:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503742592 00000100:00100000:3.0:1789760773.635319:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503742592 00000100:00100000:3.0:1789760773.635379:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703723594092 time 703743727049 state 0x0 ref 1 00000100:00100000:3.0:1789760773.635420:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703743987465 time 703743727049 state 0x1 ref 1 00000100:00100000:3.0:1789760773.635450:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703743987465 time 703760057408 state 0x0 ref 1 00000100:00100000:3.0:1789760773.635462:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503742592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760773.635509:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75358208, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f08 00010000:00010000:3.0:1789760773.635519:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.635534:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760773.635538:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.635543:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.635558:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.644367:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985705 is committed 00002000:00010000:1.0:1789760773.644506:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75358208 -> 75362304 00002000:00010000:1.0:1789760773.644512:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147208 -> 147216 00010000:00010000:1.0:1789760773.644526:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.644539:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.644548:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.644555:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.644560:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.644569:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.644589:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.644598:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083eff1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75358208->75362303] (req 75358208->75362303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.644704:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503742592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9239us (9678us total) trans 4294985705 rc 0/0 00000100:00100000:1.0:1789760773.644715:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18409 00000100:00100000:1.0:1789760773.644726:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703743987465 time 703760057408 state 0x2 ref 0 00010000:00010000:3.0:1789760773.650676:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760773.653111:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.653258:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503742720 00000100:00100000:1.0:1789760773.653269:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503742720 00000100:00100000:1.0:1789760773.653324:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703743987465 time 703760057408 state 0x0 ref 1 00000100:00100000:1.0:1789760773.653341:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703760317824 time 703760057408 state 0x1 ref 1 00000100:00100000:1.0:1789760773.653376:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703760317824 time 703777982731 state 0x0 ref 1 00000100:00100000:1.0:1789760773.653420:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503742720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.653473:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75362304, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f10 00010000:00010000:1.0:1789760773.653483:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.653501:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.653511:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.653518:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75362304->75366399] (req 75362304->75366399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.653532:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083eff22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75362304->75366399] (req 75362304->75366399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.659725:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985706 is committed 00000100:00100000:3.0:1789760773.664791:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.665100:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503742848 00000100:00100000:1.0:1789760773.665110:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503742848 00000100:00100000:1.0:1789760773.665231:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703760317824 time 703777982731 state 0x0 ref 1 00000100:00100000:1.0:1789760773.665247:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703778243147 time 703777982731 state 0x1 ref 1 00000100:00100000:1.0:1789760773.665313:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703778243147 time 703789918961 state 0x0 ref 1 00000100:00100000:1.0:1789760773.665326:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503742848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.665385:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75366400, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f18 00010000:00010000:1.0:1789760773.665426:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.665441:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.665446:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.665453:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.665467:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.671200:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985707 is committed 00002000:00010000:0.0:1789760773.671370:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75366400 -> 75370496 00002000:00010000:0.0:1789760773.671376:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147224 -> 147232 00010000:00010000:0.0:1789760773.671495:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.671510:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.671591:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.671599:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.671604:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.671623:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.671653:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.671661:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083eff29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75366400->75370495] (req 75366400->75370495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.671958:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503742848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6630us (7174us total) trans 4294985707 rc 0/0 00000100:00100000:0.0:1789760773.671969:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18411 00000100:00100000:0.0:1789760773.671979:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703778243147 time 703789918961 state 0x2 ref 0 00000100:00100000:0.0:1789760773.674784:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760773.681250:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985708 is committed 00000100:00100000:3.0:1789760773.685149:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503743104 00000100:00100000:3.0:1789760773.685159:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503743104 00000100:00100000:3.0:1789760773.685213:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703790179377 time 703799686715 state 0x0 ref 1 00000100:00100000:3.0:1789760773.685227:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703799947131 time 703799686715 state 0x1 ref 1 00000100:00100000:3.0:1789760773.685259:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703799947131 time 703809865576 state 0x0 ref 1 00000100:00100000:3.0:1789760773.685271:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503743104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760773.685315:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75374592, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f28 00010000:00010000:3.0:1789760773.685324:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.685340:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760773.685344:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.685351:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.685377:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.694451:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985709 is committed 00002000:00010000:1.0:1789760773.694685:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75374592 -> 75378688 00002000:00010000:1.0:1789760773.694691:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147240 -> 147248 00010000:00010000:1.0:1789760773.694703:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.694717:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.694726:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.694733:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.694738:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.694748:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.694767:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.694775:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083eff37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75374592->75378687] (req 75374592->75378687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.694877:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503743104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9604us (9813us total) trans 4294985709 rc 0/0 00000100:00100000:1.0:1789760773.694887:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18413 00000100:00100000:1.0:1789760773.694897:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703799947131 time 703809865576 state 0x2 ref 0 00000100:00100000:3.0:1789760773.701971:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.702076:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503743232 00000100:00100000:1.0:1789760773.702086:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503743232 00000100:00100000:1.0:1789760773.702137:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703799947131 time 703809865576 state 0x0 ref 1 00000100:00100000:1.0:1789760773.702152:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703810125992 time 703809865576 state 0x1 ref 1 00000100:00100000:1.0:1789760773.702181:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703810125992 time 703826788526 state 0x0 ref 1 00000100:00100000:1.0:1789760773.702193:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503743232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.702238:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75378688, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f30 00010000:00010000:1.0:1789760773.702247:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.702263:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.702267:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.702273:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.702287:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.709275:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985710 is committed 00002000:00010000:1.0:1789760773.709437:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75378688 -> 75382784 00002000:00010000:1.0:1789760773.709443:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147248 -> 147256 00010000:00010000:1.0:1789760773.709455:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.709466:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.709474:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.709480:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.709484:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.709493:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.709511:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.709518:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75378688->75382783] (req 75378688->75382783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.709609:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503743232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7412us (7642us total) trans 4294985710 rc 0/0 00000100:00100000:1.0:1789760773.709619:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18414 00000100:00100000:1.0:1789760773.710491:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703810125992 time 703826788526 state 0x2 ref 0 00000100:00100000:0.0:1789760773.713049:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.718421:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503743360 00000100:00100000:1.0:1789760773.718436:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503743360 00000100:00100000:1.0:1789760773.718497:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703810125992 time 703826788526 state 0x0 ref 1 00000100:00100000:1.0:1789760773.718513:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703827048942 time 703826788526 state 0x1 ref 1 00000100:00100000:1.0:1789760773.718550:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703827048942 time 703843156431 state 0x0 ref 1 00000100:00100000:1.0:1789760773.718564:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503743360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.718608:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75382784, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f38 00010000:00010000:1.0:1789760773.718617:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.718645:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.718650:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.718656:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.718679:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.726921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985711 is committed 00002000:00010000:1.0:1789760773.727068:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75382784 -> 75386880 00002000:00010000:1.0:1789760773.727072:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147256 -> 147264 00010000:00010000:1.0:1789760773.727087:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.727100:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.727109:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.727117:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.727122:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.727131:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.727149:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.727157:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75382784->75386879] (req 75382784->75386879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.727257:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503743360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8691us (14211us total) trans 4294985711 rc 0/0 00000100:00100000:1.0:1789760773.727267:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18415 00000100:00100000:1.0:1789760773.727277:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703827048942 time 703843156431 state 0x2 ref 0 00000100:00100000:0.0:1789760773.730866:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503743488 00000100:00100000:0.0:1789760773.730876:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503743488 00000100:00100000:0.0:1789760773.730927:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703827048942 time 703843156431 state 0x0 ref 1 00000100:00100000:0.0:1789760773.730942:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703843416847 time 703843156431 state 0x1 ref 1 00000100:00100000:0.0:1789760773.730970:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703843416847 time 703855577755 state 0x0 ref 1 00000100:00100000:0.0:1789760773.730981:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503743488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.731022:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75386880, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f40 00010000:00010000:0.0:1789760773.731030:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.731045:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.731049:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.731055:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.731068:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.737428:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985712 is committed 00002000:00010000:0.0:1789760773.737590:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75386880 -> 75390976 00002000:00010000:0.0:1789760773.737595:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147264 -> 147272 00010000:00010000:0.0:1789760773.737608:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737621:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737647:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737655:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.737659:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737864:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737885:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.737892:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75386880->75390975] (req 75386880->75390975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.738015:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503743488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7031us (7243us total) trans 4294985712 rc 0/0 00000100:00100000:0.0:1789760773.738025:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18416 00000100:00100000:0.0:1789760773.738034:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703843416847 time 703855577755 state 0x2 ref 0 00000100:00100000:1.0:1789760773.743373:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760773.743497:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503743616 00000100:00100000:3.0:1789760773.743508:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503743616 00000100:00100000:3.0:1789760773.743572:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703843416847 time 703855577755 state 0x0 ref 1 00000100:00100000:3.0:1789760773.743587:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703855838171 time 703855577755 state 0x1 ref 1 00000100:00100000:3.0:1789760773.743796:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703855838171 time 703868401486 state 0x0 ref 1 00000100:00100000:3.0:1789760773.743811:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503743616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760773.743858:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75390976, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f48 00010000:00010000:3.0:1789760773.743868:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.743884:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760773.743889:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.743895:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.743909:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.756296:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985713 is committed 00002000:00010000:3.0:1789760773.757525:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75390976 -> 75395072 00002000:00010000:3.0:1789760773.757532:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147272 -> 147280 00010000:00010000:3.0:1789760773.757546:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.757559:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.757567:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.757574:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760773.757579:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.757588:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.757607:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.757616:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083eff53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75390976->75395071] (req 75390976->75395071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.757780:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503743616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13967us (14411us total) trans 4294985713 rc 0/0 00000100:00100000:3.0:1789760773.757791:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18417 00000100:00100000:3.0:1789760773.757800:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703855838171 time 703868401486 state 0x2 ref 0 00000100:00100000:3.0:1789760773.764117:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.764373:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503743744 00000100:00100000:1.0:1789760773.764384:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503743744 00000100:00100000:1.0:1789760773.764480:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703855838171 time 703868401486 state 0x0 ref 1 00000100:00100000:1.0:1789760773.764498:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703868661902 time 703868401486 state 0x1 ref 1 00000100:00100000:1.0:1789760773.764518:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703868661902 time 703889125277 state 0x0 ref 1 00000100:00100000:1.0:1789760773.764530:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503743744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.764580:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75395072, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f50 00010000:00010000:1.0:1789760773.764589:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.764605:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.764610:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.764616:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.764637:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.771449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985714 is committed 00002000:00010000:1.0:1789760773.771557:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75395072 -> 75399168 00002000:00010000:1.0:1789760773.771562:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147280 -> 147288 00010000:00010000:1.0:1789760773.771574:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.771587:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.771605:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.771612:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.771617:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.771634:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.771655:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.771662:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75395072->75399167] (req 75395072->75399167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.771762:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503743744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7229us (7652us total) trans 4294985714 rc 0/0 00000100:00100000:1.0:1789760773.771772:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18418 00000100:00100000:1.0:1789760773.771781:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703868661902 time 703889125277 state 0x2 ref 0 00000100:00100000:1.0:1789760773.774635:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503743872 00000100:00100000:1.0:1789760773.774646:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503743872 00000100:00100000:1.0:1789760773.774710:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703868661902 time 703889125277 state 0x0 ref 1 00000100:00100000:1.0:1789760773.774726:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703889385693 time 703889125277 state 0x1 ref 1 00000100:00100000:1.0:1789760773.774754:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703889385693 time 703899360652 state 0x0 ref 1 00000100:00100000:1.0:1789760773.774767:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503743872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.774813:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75399168, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x23f58 00010000:00010000:1.0:1789760773.774822:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.774837:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.774841:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.774847:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.774862:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.780242:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985715 is committed 00002000:00010000:1.0:1789760773.780642:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75399168 -> 75403264 00002000:00010000:1.0:1789760773.780648:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:1.0:1789760773.780650:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00002000:00010000:1.0:1789760773.780653:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147288 -> 147296 00010000:00010000:1.0:1789760773.780768:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.780788:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.780797:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.780806:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.780810:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.780820:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.780841:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.780848:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75399168->75403263] (req 75399168->75403263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.780938:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503743872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6169us (6459us total) trans 4294985715 rc 0/0 00000100:00100000:1.0:1789760773.780947:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18419 00000100:00100000:1.0:1789760773.780958:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703889385693 time 703899360652 state 0x2 ref 0 00000100:00100000:3.0:1789760773.783903:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.784267:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744000 00000100:00100000:0.0:1789760773.784277:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744000 00000100:00100000:0.0:1789760773.784334:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703889385693 time 703899360652 state 0x0 ref 1 00000100:00100000:0.0:1789760773.784358:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703899621068 time 703899360652 state 0x1 ref 1 00000100:00100000:0.0:1789760773.784386:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703899621068 time 703908993240 state 0x0 ref 1 00000100:00100000:0.0:1789760773.784420:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc19050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.784466:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75403264, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f60 00010000:00010000:0.0:1789760773.784475:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.784490:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.784494:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.784500:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.784514:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.788922:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985716 is committed 00002000:00010000:0.0:1789760773.791334:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75403264 -> 75407360 00002000:00010000:0.0:1789760773.791345:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147296 -> 147304 00010000:00010000:0.0:1789760773.791454:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.791468:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.791478:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.791492:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.791497:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.791507:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.791526:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.791533:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75403264->75407359] (req 75403264->75407359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.791636:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc19050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7204us (7728us total) trans 4294985716 rc 0/0 00000100:00100000:0.0:1789760773.791648:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18420 00000100:00100000:0.0:1789760773.791659:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703899621068 time 703908993240 state 0x2 ref 0 00000100:00100000:3.0:1789760773.793995:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.794088:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744128 00000100:00100000:1.0:1789760773.794098:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744128 00000100:00100000:1.0:1789760773.794150:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703899621068 time 703908993240 state 0x0 ref 1 00000100:00100000:1.0:1789760773.794165:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703909253656 time 703908993240 state 0x1 ref 1 00000100:00100000:1.0:1789760773.794198:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703909253656 time 703918804774 state 0x0 ref 1 00000100:00100000:1.0:1789760773.794211:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503744128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.794258:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75407360, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f68 00010000:00010000:1.0:1789760773.794267:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.794283:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.794287:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.794293:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.794306:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.800285:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75407360 -> 75411456 00002000:00010000:1.0:1789760773.800291:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147304 -> 147312 00010000:00010000:1.0:1789760773.800303:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800315:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800328:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800335:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.800339:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800347:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800491:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800499:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083eff6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75407360->75411455] (req 75407360->75411455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.800634:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503744128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6411us (6633us total) trans 4294985717 rc 0/0 00000100:00100000:1.0:1789760773.800645:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18421 00000100:00100000:1.0:1789760773.800655:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703909253656 time 703918804774 state 0x2 ref 0 00000100:00100000:1.0:1789760773.807056:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744256 00000100:00100000:1.0:1789760773.807073:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744256 00000100:00100000:1.0:1789760773.807155:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703909253656 time 703918804774 state 0x0 ref 1 00000100:00100000:1.0:1789760773.807175:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703919065190 time 703918804774 state 0x1 ref 1 00000100:00100000:1.0:1789760773.807262:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703919065190 time 703931867258 state 0x0 ref 1 00000100:00100000:1.0:1789760773.807278:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.807346:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75411456, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f70 00010000:00010000:1.0:1789760773.807368:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.807423:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.807429:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.807438:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75411456->75415551] (req 75411456->75415551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.807455:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75411456->75415551] (req 75411456->75415551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.813124:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985718 is committed 00000100:00100000:0.0:1789760773.818832:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744384 00000100:00100000:0.0:1789760773.818845:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744384 00000100:00100000:0.0:1789760773.818921:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703919065190 time 703931867258 state 0x0 ref 1 00000100:00100000:0.0:1789760773.818940:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703932127674 time 703931867258 state 0x1 ref 1 00000100:00100000:0.0:1789760773.819040:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703932127674 time 703943646225 state 0x0 ref 1 00000100:00100000:0.0:1789760773.819060:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.819124:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75415552, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f78 00010000:00010000:0.0:1789760773.819135:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.819154:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.819159:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.819167:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.819183:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.829783:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985719 is committed 00002000:00010000:1.0:1789760773.830049:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75415552 -> 75419648 00002000:00010000:1.0:1789760773.830056:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147320 -> 147328 00010000:00010000:1.0:1789760773.830073:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.830156:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.830167:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.830182:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.830188:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.830199:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.830223:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.830231:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75415552->75419647] (req 75415552->75419647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.830596:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11530us (11861us total) trans 4294985719 rc 0/0 00000100:00100000:1.0:1789760773.830609:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18423 00000100:00100000:1.0:1789760773.830623:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703932127674 time 703943646225 state 0x2 ref 0 00000100:00100000:3.0:1789760773.834326:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.834926:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744512 00000100:00100000:0.0:1789760773.834940:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744512 00000100:00100000:0.0:1789760773.835015:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703932127674 time 703943646225 state 0x0 ref 1 00000100:00100000:0.0:1789760773.835034:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703943906641 time 703943646225 state 0x1 ref 1 00000100:00100000:0.0:1789760773.835116:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703943906641 time 703959721899 state 0x0 ref 1 00000100:00100000:0.0:1789760773.835131:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503744512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.835201:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75419648, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f80 00010000:00010000:0.0:1789760773.835213:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.835237:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760773.835242:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.835250:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.835266:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.842586:0:3637:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760773.842616:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6739500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523606144:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760773.842712:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760773.842808:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606144 00000100:00100000:1.0:1789760773.842817:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606144 00000100:00100000:1.0:1789760773.842881:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789760773.842891:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523606144:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760773.842934:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1789760773.842980:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523606144:12345-0@lo:13:osp-pre-1-1.0 Request processed in 89us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760773.842987:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00080000:1.0:1789760773.843007:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760773.843047:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6739500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523606144:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760773.843265:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc687bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523606272:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760773.843291:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760773.843370:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606272 00000100:00100000:3.0:1789760773.843375:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606272 00000100:00100000:3.0:1789760773.843440:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789760773.843447:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523606272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760773.843460:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760773.843490:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523606272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 43us (198us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.843496:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00080000:1.0:1789760773.843597:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760773.843621:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc687bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523606272:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760773.844550:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc687b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:afc717a1-11ba-4ef0-ab10-103aa692ca98:3638:1876699523606400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.844603:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760773.844686:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606400 00000100:00100000:1.0:1789760773.844692:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606400 00000100:00100000:1.0:1789760773.844734:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:1.0:1789760773.844742:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3638:x1876699523606400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845468:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6878e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3639:1876699523607040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845500:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760773.845517:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ccea0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3639:1876699523607552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845534:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760773.845549:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789760773.845554:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc687b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523606912:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845575:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760773.845588:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ccea0d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523607424:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845608:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760773.845643:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606528 00000100:00100000:3.0:1789760773.845648:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606528 00000100:00100000:3.0:1789760773.845680:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607040 00000100:00100000:3.0:1789760773.845682:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607040 00000100:00100000:3.0:1789760773.845704:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607552 00000100:00100000:3.0:1789760773.845706:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607552 00000100:00100000:3.0:1789760773.845720:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 955 00000100:00100000:3.0:1789760773.845725:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3639:x1876699523606528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845774:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3639:x1876699523606528:12345-0@lo:400:kworker.0 Request processed in 48us (488us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.845782:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 955 00000100:00100000:3.0:1789760773.845799:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 956 00000100:00100000:3.0:1789760773.845805:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f70700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3639:x1876699523607040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845826:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f70700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3639:x1876699523607040:12345-0@lo:400:kworker.0 Request processed in 22us (326us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.845832:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 956 00000100:00100000:3.0:1789760773.845841:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 957 00000100:00100000:3.0:1789760773.845845:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f70a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3639:x1876699523607552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845865:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f70a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3639:x1876699523607552:12345-0@lo:400:kworker.0 Request processed in 19us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.845870:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 957 00000100:00100000:3.0:1789760773.845884:0:8490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607424 00000100:00100000:3.0:1789760773.845887:0:8490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607424 00000100:00100000:3.0:1789760773.845911:0:8490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1789760773.845917:0:8490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523607424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845942:0:8490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523607424:12345-0@lo:400:kworker.0 Request processed in 25us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.845948:0:8490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1789760773.845982:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523607424:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845991:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6878e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3639:1876699523607040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.845998:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3639:1876699523607552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.846006:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc687aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3639:1876699523606528:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.846357:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc687bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523606656:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.846417:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760773.846443:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ccea0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523607168:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.846469:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760773.846491:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc8f73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523607680:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.846516:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760773.846558:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606912 00000100:00100000:0.0:1789760773.846577:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606912 00000100:00100000:0.0:1789760773.846638:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606656 00000100:00100000:0.0:1789760773.846643:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606656 00000100:00100000:0.0:1789760773.846800:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:0.0:1789760773.846809:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523606912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.847062:0:8429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607296 00000100:00100000:3.0:1789760773.847067:0:8429:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607296 00000100:00100000:3.0:1789760773.847318:0:8429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:3.0:1789760773.847327:0:8429:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523607296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760773.849908:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3638:x1876699523606400:12345-0@lo:400:kworker.0 Request processed in 5156us (5294us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760773.849924:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:0.0:1789760773.849964:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523606912:12345-0@lo:400:kworker.0 Request processed in 3155us (4385us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.849973:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:0.0:1789760773.850023:0:8237:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607168 00000100:00100000:0.0:1789760773.850034:0:8237:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607168 00000100:00100000:0.0:1789760773.850077:0:8237:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607680 00000100:00100000:0.0:1789760773.850086:0:8237:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607680 00000100:00100000:0.0:1789760773.850102:0:8237:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523606784 00000100:00100000:0.0:1789760773.850105:0:8237:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523606784 00000100:00100000:0.0:1789760773.850120:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 958 00000100:00100000:0.0:1789760773.850128:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3636:x1876699523607168:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.850198:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3636:x1876699523607168:12345-0@lo:400:kworker.0 Request processed in 71us (3728us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.850206:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 958 00000100:00100000:0.0:1789760773.850218:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 959 00000100:00100000:0.0:1789760773.850224:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3636:x1876699523607680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.850257:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3636:x1876699523607680:12345-0@lo:400:kworker.0 Request processed in 33us (3740us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.850263:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 959 00000100:00100000:0.0:1789760773.850291:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 960 00000100:00100000:0.0:1789760773.850298:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3637:x1876699523606784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.850338:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3637:x1876699523606784:12345-0@lo:400:kworker.0 Request processed in 40us (3740us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.850343:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 960 00000100:00100000:0.0:1789760773.850433:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523607168:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.850451:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8f73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523607680:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.850516:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc687b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523606912:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.850528:0:8429:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523607296:12345-0@lo:400:kworker.0 Request processed in 3200us (3875us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.850543:0:8429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00002000:00010000:1.0:1789760773.850555:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75419648 -> 75423744 00000100:00100000:0.0:1789760773.850556:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc687b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3637:1876699523606784:0@lo:400:kworker.0 00002000:00010000:1.0:1789760773.850561:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147328 -> 147336 00010000:00010000:1.0:1789760773.850580:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.850595:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760773.850601:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:1.0:1789760773.850606:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760773.850609:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760773.850615:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789760773.850620:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.850637:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.850649:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.850673:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.850682:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75419648->75423743] (req 75419648->75423743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.850758:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503744512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15627us (16440us total) trans 4294985720 rc 0/0 00000100:00100000:1.0:1789760773.850768:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18424 00000100:00100000:1.0:1789760773.850779:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703943906641 time 703959721899 state 0x2 ref 0 00000100:00100000:0.0:1789760773.851323:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523607296:0@lo:400:kworker.0 00000100:00100000:1.0:1789760773.851923:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc687bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523606656:0@lo:400:kworker.0 00000100:00100000:1.0:1789760773.855521:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744640 00000100:00100000:1.0:1789760773.855533:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744640 00000100:00100000:1.0:1789760773.855605:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703943906641 time 703959721899 state 0x0 ref 1 00000100:00100000:1.0:1789760773.855624:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703959982315 time 703959721899 state 0x1 ref 1 00000100:00100000:1.0:1789760773.856834:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703959982315 time 703980272577 state 0x0 ref 1 00000100:00100000:1.0:1789760773.856870:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503744640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.856942:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75423744, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f88 00010000:00010000:1.0:1789760773.856953:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.856974:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760773.856979:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.856987:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.857005:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.869099:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985721 is committed 00002000:00010000:1.0:1789760773.869493:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75423744 -> 75427840 00002000:00010000:1.0:1789760773.869642:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147336 -> 147344 00010000:00010000:1.0:1789760773.869662:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.869822:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.869835:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.869851:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.869858:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.869870:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.869897:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.869906:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083eff8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75423744->75427839] (req 75423744->75427839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.870044:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503744640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13178us (15069us total) trans 4294985721 rc 0/0 00000100:00100000:1.0:1789760773.870057:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18425 00000100:00100000:1.0:1789760773.870070:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703959982315 time 703980272577 state 0x2 ref 0 00000100:00100000:1.0:1789760773.882258:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744768 00000100:00100000:1.0:1789760773.882280:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744768 00000100:00100000:0.0:1789760773.882561:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703959982315 time 703980272577 state 0x0 ref 1 00000100:00100000:0.0:1789760773.882581:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703980532993 time 703980272577 state 0x1 ref 1 00000100:00100000:0.0:1789760773.882675:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703980532993 time 704007272506 state 0x0 ref 1 00000100:00100000:0.0:1789760773.882690:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.882757:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75427840, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f90 00010000:00010000:0.0:1789760773.882768:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.882788:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760773.882794:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.882801:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.882822:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.889071:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985722 is committed 00002000:00010000:0.0:1789760773.889280:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75427840 -> 75431936 00002000:00010000:0.0:1789760773.889286:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147344 -> 147352 00010000:00010000:0.0:1789760773.889304:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889323:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889332:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889340:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.889345:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889365:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889411:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889420:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083eff92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75427840->75431935] (req 75427840->75431935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.889540:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6846us (10186us total) trans 4294985722 rc 0/0 00000100:00100000:0.0:1789760773.889553:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18426 00000100:00100000:0.0:1789760773.889565:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703980532993 time 704007272506 state 0x2 ref 0 00000100:00100000:3.0:1789760773.894148:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.894253:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503744896 00000100:00100000:0.0:1789760773.894267:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503744896 00000100:00100000:0.0:1789760773.894338:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 703980532993 time 704007272506 state 0x0 ref 1 00000100:00100000:0.0:1789760773.894369:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704007532922 time 704007272506 state 0x1 ref 1 00000100:00100000:0.0:1789760773.894447:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704007532922 time 704019053429 state 0x0 ref 1 00000100:00100000:0.0:1789760773.894462:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.894529:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75431936, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23f98 00010000:00010000:0.0:1789760773.894541:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.894561:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760773.894566:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.894574:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.894590:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.903646:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75431936 -> 75436032 00002000:00010000:0.0:1789760773.903660:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147352 -> 147360 00010000:00010000:0.0:1789760773.903680:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.903696:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.903781:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.903790:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.903797:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.903808:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.903836:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.903845:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083eff99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75431936->75436031] (req 75431936->75436031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.903979:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503744896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9512us (9838us total) trans 4294985723 rc 0/0 00000100:00100000:0.0:1789760773.903991:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18427 00000100:00100000:0.0:1789760773.904004:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704007532922 time 704019053429 state 0x2 ref 0 00000100:00100000:1.0:1789760773.910599:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745024 00000100:00100000:1.0:1789760773.910610:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745024 00000100:00100000:1.0:1789760773.910934:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704007532922 time 704019053429 state 0x0 ref 1 00000100:00100000:1.0:1789760773.910954:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704019313845 time 704019053429 state 0x1 ref 1 00000100:00100000:1.0:1789760773.911057:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704019313845 time 704035659963 state 0x0 ref 1 00000100:00100000:1.0:1789760773.911074:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503745024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.911147:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75436032, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fa0 00010000:00010000:1.0:1789760773.911160:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.911183:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760773.911189:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.911196:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.911213:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.919378:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985724 is committed 00002000:00010000:1.0:1789760773.919551:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75436032 -> 75440128 00002000:00010000:1.0:1789760773.919558:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147360 -> 147368 00010000:00010000:1.0:1789760773.919575:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.919591:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.919601:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.919615:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.919620:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921251:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921285:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.921295:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75436032->75440127] (req 75436032->75440127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.921993:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503745024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10915us (11477us total) trans 4294985724 rc 0/0 00000100:00100000:1.0:1789760773.922008:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18428 00000100:00100000:1.0:1789760773.922021:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704019313845 time 704035659963 state 0x2 ref 0 00000100:00100000:1.0:1789760773.928481:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745152 00000100:00100000:1.0:1789760773.928494:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745152 00000100:00100000:1.0:1789760773.928573:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704019313845 time 704035659963 state 0x0 ref 1 00000100:00100000:1.0:1789760773.928592:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704035920379 time 704035659963 state 0x1 ref 1 00000100:00100000:1.0:1789760773.928641:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704035920379 time 704053246825 state 0x0 ref 1 00000100:00100000:1.0:1789760773.928660:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503745152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.928740:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75440128, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fa8 00010000:00010000:1.0:1789760773.928760:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.928780:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760773.928786:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.928804:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.928823:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.935523:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985725 is committed 00002000:00010000:1.0:1789760773.935687:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75440128 -> 75444224 00002000:00010000:1.0:1789760773.935693:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147368 -> 147376 00010000:00010000:1.0:1789760773.935716:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.935732:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.935742:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.935759:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.935766:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.935776:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.935810:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.935818:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effa7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75440128->75444223] (req 75440128->75444223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.935940:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503745152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7275us (7603us total) trans 4294985725 rc 0/0 00000100:00100000:1.0:1789760773.935962:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18429 00000100:00100000:1.0:1789760773.935976:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704035920379 time 704053246825 state 0x2 ref 0 00000100:00100000:3.0:1789760773.941231:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.941792:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745280 00000100:00100000:0.0:1789760773.941806:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745280 00000100:00100000:0.0:1789760773.941885:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704035920379 time 704053246825 state 0x0 ref 1 00000100:00100000:0.0:1789760773.941905:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704053507241 time 704053246825 state 0x1 ref 1 00000100:00100000:0.0:1789760773.941937:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704053507241 time 704066542796 state 0x0 ref 1 00000100:00100000:0.0:1789760773.941951:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503745280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.942016:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75444224, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fb0 00010000:00010000:0.0:1789760773.942027:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.942048:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760773.942053:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.942061:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.942076:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.951167:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75444224 -> 75448320 00002000:00010000:0.0:1789760773.951174:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147376 -> 147384 00010000:00010000:0.0:1789760773.951190:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.951206:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.951215:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.951224:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.951229:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.951244:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.951269:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.951277:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75444224->75448319] (req 75444224->75448319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.951449:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503745280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9494us (10231us total) trans 4294985726 rc 0/0 00000100:00100000:0.0:1789760773.951462:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18430 00000100:00100000:0.0:1789760773.951473:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704053507241 time 704066542796 state 0x2 ref 0 00000100:00100000:3.0:1789760773.958552:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760773.958684:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745408 00000100:00100000:0.0:1789760773.958697:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745408 00000100:00100000:0.0:1789760773.958787:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704053507241 time 704066542796 state 0x0 ref 1 00000100:00100000:0.0:1789760773.958814:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704066803212 time 704066542796 state 0x1 ref 1 00000100:00100000:0.0:1789760773.958849:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704066803212 time 704083455270 state 0x0 ref 1 00000100:00100000:0.0:1789760773.958867:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503745408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.958931:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75448320, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fb8 00010000:00010000:0.0:1789760773.958951:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.958972:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760773.958978:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.958986:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.959004:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.967360:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75448320 -> 75452416 00002000:00010000:0.0:1789760773.967367:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147384 -> 147392 00010000:00010000:0.0:1789760773.967386:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967447:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967457:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967465:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760773.967471:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967482:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967518:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967526:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75448320->75452415] (req 75448320->75452415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.967654:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503745408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8782us (9110us total) trans 4294985727 rc 0/0 00000100:00100000:0.0:1789760773.967666:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18431 00000100:00100000:0.0:1789760773.967679:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704066803212 time 704083455270 state 0x2 ref 0 00000100:00100000:1.0:1789760773.976515:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745536 00000100:00100000:1.0:1789760773.976529:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745536 00000100:00100000:1.0:1789760773.976606:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704066803212 time 704083455270 state 0x0 ref 1 00000100:00100000:1.0:1789760773.976634:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704083715686 time 704083455270 state 0x1 ref 1 00000100:00100000:1.0:1789760773.976670:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704083715686 time 704101275415 state 0x0 ref 1 00000100:00100000:1.0:1789760773.976685:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503745536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760773.976912:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75452416, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fc0 00010000:00010000:1.0:1789760773.976923:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.977006:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760773.977013:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.977022:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.977039:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.982994:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75452416 -> 75456512 00002000:00010000:1.0:1789760773.983000:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147392 -> 147400 00010000:00010000:1.0:1789760773.983018:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.983033:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.983045:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.983054:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760773.983059:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.983070:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.983106:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.983115:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75452416->75456511] (req 75452416->75456511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.983246:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503745536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6556us (7307us total) trans 4294985728 rc 0/0 00000100:00100000:1.0:1789760773.983257:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18432 00000100:00100000:1.0:1789760773.983271:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704083715686 time 704101275415 state 0x2 ref 0 00000100:00100000:3.0:1789760773.989063:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760773.989887:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745664 00000100:00100000:1.0:1789760773.989906:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745664 00000100:00100000:0.0:1789760773.990155:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704083715686 time 704101275415 state 0x0 ref 1 00000100:00100000:0.0:1789760773.990253:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704101535831 time 704101275415 state 0x1 ref 1 00000100:00100000:0.0:1789760773.990339:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704101535831 time 704114943616 state 0x0 ref 1 00000100:00100000:0.0:1789760773.990355:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503745664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760773.990452:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75456512, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fc8 00010000:00010000:0.0:1789760773.990464:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.990485:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760773.990490:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.990498:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.990515:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.999711:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985729 is committed 00002000:00010000:3.0:1789760774.002207:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75456512 -> 75460608 00002000:00010000:3.0:1789760774.002219:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147400 -> 147408 00010000:00010000:3.0:1789760774.002238:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.002260:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.002596:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.002605:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760774.002610:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.002621:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.002656:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.002664:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083effc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75456512->75460607] (req 75456512->75460607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.002791:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503745664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12432us (13742us total) trans 4294985729 rc 0/0 00000100:00100000:3.0:1789760774.002804:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18433 00000100:00100000:3.0:1789760774.002817:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704101535831 time 704114943616 state 0x2 ref 0 00000100:00100000:3.0:1789760774.007222:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.007773:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745792 00000100:00100000:0.0:1789760774.007786:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745792 00000100:00100000:0.0:1789760774.007883:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704101535831 time 704114943616 state 0x0 ref 1 00000100:00100000:0.0:1789760774.007902:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704115204032 time 704114943616 state 0x1 ref 1 00000100:00100000:0.0:1789760774.007933:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704115204032 time 704132539388 state 0x0 ref 1 00000100:00100000:0.0:1789760774.007948:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503745792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.008011:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75460608, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fd0 00010000:00010000:0.0:1789760774.008023:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.008043:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.008048:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.008055:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.008071:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760774.008533:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760774.016670:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75460608 -> 75464704 00002000:00010000:0.0:1789760774.016685:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147408 -> 147416 00010000:00010000:0.0:1789760774.016703:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.016719:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.016730:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.016738:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.016744:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.017178:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.017656:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.017671:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75460608->75464703] (req 75460608->75464703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.017896:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503745792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9942us (10680us total) trans 4294985730 rc 0/0 00000100:00100000:0.0:1789760774.017912:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18434 00000100:00100000:0.0:1789760774.017927:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704115204032 time 704132539388 state 0x2 ref 0 00000100:00100000:1.0:1789760774.024108:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503745920 00000100:00100000:1.0:1789760774.024122:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503745920 00000100:00100000:1.0:1789760774.024198:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704115204032 time 704132539388 state 0x0 ref 1 00000100:00100000:1.0:1789760774.024217:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704132799804 time 704132539388 state 0x1 ref 1 00000100:00100000:1.0:1789760774.024293:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704132799804 time 704148898320 state 0x0 ref 1 00000100:00100000:1.0:1789760774.024309:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503745920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.024386:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75464704, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fd8 00010000:00010000:1.0:1789760774.024430:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.024455:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.024461:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.024468:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.024491:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.032071:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985731 is committed 00002000:00010000:1.0:1789760774.034875:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75464704 -> 75468800 00002000:00010000:1.0:1789760774.034890:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147416 -> 147424 00010000:00010000:1.0:1789760774.034911:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.034926:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035075:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035085:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.035090:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035101:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035133:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035141:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75464704->75468799] (req 75464704->75468799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.035362:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503745920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11047us (11369us total) trans 4294985731 rc 0/0 00000100:00100000:1.0:1789760774.035375:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18435 00000100:00100000:1.0:1789760774.035436:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704132799804 time 704148898320 state 0x2 ref 0 00000100:00100000:3.0:1789760774.045979:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.046195:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746048 00000100:00100000:1.0:1789760774.046210:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746048 00000100:00100000:1.0:1789760774.046289:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704132799804 time 704148898320 state 0x0 ref 1 00000100:00100000:1.0:1789760774.046308:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704149158736 time 704148898320 state 0x1 ref 1 00000100:00100000:1.0:1789760774.046348:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704149158736 time 704170953716 state 0x0 ref 1 00000100:00100000:1.0:1789760774.046363:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503746048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.046579:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75468800, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fe0 00010000:00010000:1.0:1789760774.046592:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.046611:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.046617:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.046636:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.046831:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.051847:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985732 is committed 00002000:00010000:1.0:1789760774.054149:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75468800 -> 75472896 00002000:00010000:1.0:1789760774.054164:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147424 -> 147432 00010000:00010000:1.0:1789760774.054183:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.054268:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.054279:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.054288:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.054295:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.054535:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.054562:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.054577:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75468800->75472895] (req 75468800->75472895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.054856:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503746048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8485us (8880us total) trans 4294985732 rc 0/0 00000100:00100000:1.0:1789760774.054868:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18436 00000100:00100000:1.0:1789760774.054880:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704149158736 time 704170953716 state 0x2 ref 0 00000100:00100000:3.0:1789760774.061755:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.061884:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746176 00000100:00100000:0.0:1789760774.061896:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746176 00000100:00100000:0.0:1789760774.061972:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704149158736 time 704170953716 state 0x0 ref 1 00000100:00100000:0.0:1789760774.061992:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704171214132 time 704170953716 state 0x1 ref 1 00000100:00100000:0.0:1789760774.062119:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704171214132 time 704186724111 state 0x0 ref 1 00000100:00100000:0.0:1789760774.062145:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503746176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.062206:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75472896, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23fe8 00010000:00010000:0.0:1789760774.062218:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.062237:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.062242:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.062250:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.062272:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.070798:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75472896 -> 75476992 00002000:00010000:0.0:1789760774.070805:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147432 -> 147440 00010000:00010000:0.0:1789760774.070823:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.070837:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.070846:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.070853:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.070860:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.070870:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.070895:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.070902:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effdf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75472896->75476991] (req 75472896->75476991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.071046:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503746176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8897us (9297us total) trans 4294985733 rc 0/0 00000100:00100000:0.0:1789760774.071060:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18437 00000100:00100000:0.0:1789760774.071073:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704171214132 time 704186724111 state 0x2 ref 0 00000100:00100000:0.0:1789760774.076904:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746304 00000100:00100000:0.0:1789760774.076917:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746304 00000100:00100000:0.0:1789760774.077002:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704171214132 time 704186724111 state 0x0 ref 1 00000100:00100000:0.0:1789760774.077022:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704186984527 time 704186724111 state 0x1 ref 1 00000100:00100000:0.0:1789760774.077057:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704186984527 time 704201663185 state 0x0 ref 1 00000100:00100000:0.0:1789760774.077071:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503746304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.077426:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75476992, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23ff0 00010000:00010000:0.0:1789760774.077438:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.077460:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.077467:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.077475:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.077491:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.084673:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75476992 -> 75481088 00002000:00010000:0.0:1789760774.084679:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147440 -> 147448 00010000:00010000:0.0:1789760774.084697:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.084723:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.084734:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.084742:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.084748:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.084762:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.084789:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.084800:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083effe6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75476992->75481087] (req 75476992->75481087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.084932:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503746304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7856us (8340us total) trans 4294985734 rc 0/0 00000100:00100000:0.0:1789760774.085058:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18438 00000100:00100000:0.0:1789760774.085075:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704186984527 time 704201663185 state 0x2 ref 0 00000100:00100000:3.0:1789760774.098906:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.099317:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746432 00000100:00100000:1.0:1789760774.099331:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746432 00000100:00100000:1.0:1789760774.099944:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704186984527 time 704201663185 state 0x0 ref 1 00000100:00100000:1.0:1789760774.099965:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704201923601 time 704201663185 state 0x1 ref 1 00000100:00100000:1.0:1789760774.100005:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704201923601 time 704224611519 state 0x0 ref 1 00000100:00100000:1.0:1789760774.100023:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503746432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.100093:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75481088, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x23ff8 00010000:00010000:1.0:1789760774.100105:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.100125:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.100131:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.100138:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.100161:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.108540:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75481088 -> 75485184 00002000:00010000:1.0:1789760774.108547:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147448 -> 147456 00010000:00010000:1.0:1789760774.108564:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.108581:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.108590:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.108599:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.108605:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.109684:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.109714:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.109723:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083effed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75481088->75485183] (req 75481088->75485183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.109915:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503746432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9890us (11019us total) trans 4294985735 rc 0/0 00000100:00100000:1.0:1789760774.109927:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18439 00000100:00100000:1.0:1789760774.109940:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704201923601 time 704224611519 state 0x2 ref 0 00000100:00100000:3.0:1789760774.115539:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.115798:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746560 00000100:00100000:1.0:1789760774.115813:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746560 00000100:00100000:1.0:1789760774.116273:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704201923601 time 704224611519 state 0x0 ref 1 00000100:00100000:1.0:1789760774.116296:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704224871935 time 704224611519 state 0x1 ref 1 00000100:00100000:1.0:1789760774.116419:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704224871935 time 704241024321 state 0x0 ref 1 00000100:00100000:1.0:1789760774.116437:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503746560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.116886:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75485184, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24000 00010000:00010000:1.0:1789760774.116899:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.116921:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.116926:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.116934:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.116960:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.123773:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75485184 -> 75489280 00002000:00010000:1.0:1789760774.123781:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147456 -> 147464 00010000:00010000:1.0:1789760774.123800:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.123816:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.123826:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.123835:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.123841:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.123853:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.123880:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.123889:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083efff4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75485184->75489279] (req 75485184->75489279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.124136:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503746560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7695us (8605us total) trans 4294985736 rc 0/0 00000100:00100000:1.0:1789760774.124150:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18440 00000100:00100000:1.0:1789760774.124163:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704224871935 time 704241024321 state 0x2 ref 0 00000100:00100000:0.0:1789760774.134132:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746688 00000100:00100000:0.0:1789760774.134152:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746688 00000100:00100000:0.0:1789760774.134320:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704224871935 time 704241024321 state 0x0 ref 1 00000100:00100000:0.0:1789760774.134341:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704241284737 time 704241024321 state 0x1 ref 1 00000100:00100000:0.0:1789760774.134368:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704241284737 time 704258975191 state 0x0 ref 1 00000100:00100000:0.0:1789760774.134384:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503746688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.134583:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75489280, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24008 00010000:00010000:0.0:1789760774.134596:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.134635:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.134641:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.134648:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.134664:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.141317:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75489280 -> 75493376 00002000:00010000:0.0:1789760774.141324:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147464 -> 147472 00010000:00010000:0.0:1789760774.141343:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.141366:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.141376:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.141385:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.141412:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.141423:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.141449:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.141457:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083efffb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75489280->75493375] (req 75489280->75493375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.141664:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503746688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7276us (8690us total) trans 4294985737 rc 0/0 00000100:00100000:0.0:1789760774.141678:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18441 00000100:00100000:0.0:1789760774.141691:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704241284737 time 704258975191 state 0x2 ref 0 00000100:00100000:0.0:1789760774.149980:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746816 00000100:00100000:0.0:1789760774.149994:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746816 00000100:00100000:0.0:1789760774.150065:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704241284737 time 704258975191 state 0x0 ref 1 00000100:00100000:0.0:1789760774.150085:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704259235607 time 704258975191 state 0x1 ref 1 00000100:00100000:0.0:1789760774.150125:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704259235607 time 704274731033 state 0x0 ref 1 00000100:00100000:0.0:1789760774.150139:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503746816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.150204:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75493376, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24010 00010000:00010000:0.0:1789760774.150215:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.150236:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.150253:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.150265:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.150283:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.158480:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985738 is committed 00002000:00010000:1.0:1789760774.158708:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75493376 -> 75497472 00002000:00010000:1.0:1789760774.158713:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147472 -> 147480 00010000:00010000:1.0:1789760774.158725:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.158736:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.158745:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.158753:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.158757:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.158766:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.158786:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.158794:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0002 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75493376->75497471] (req 75493376->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.158897:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503746816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8756us (9133us total) trans 4294985738 rc 0/0 00000100:00100000:1.0:1789760774.158907:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18442 00000100:00100000:1.0:1789760774.158978:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704259235607 time 704274731033 state 0x2 ref 0 00000100:00100000:0.0:1789760774.164805:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.164996:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503746944 00000100:00100000:1.0:1789760774.165006:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503746944 00000100:00100000:1.0:1789760774.165062:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704259235607 time 704274731033 state 0x0 ref 1 00000100:00100000:1.0:1789760774.165077:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704274991449 time 704274731033 state 0x1 ref 1 00000100:00100000:1.0:1789760774.165115:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704274991449 time 704289721846 state 0x0 ref 1 00000100:00100000:1.0:1789760774.165128:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ece300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503746944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.165175:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75497472, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24018 00010000:00010000:1.0:1789760774.165184:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.165199:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.165204:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.165210:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.165224:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.181244:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985739 is committed 00002000:00010000:0.0:1789760774.187169:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75497472 -> 75501568 00002000:00010000:0.0:1789760774.187178:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147480 -> 147488 00010000:00010000:0.0:1789760774.187190:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.187202:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.187210:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.187218:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.187223:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.187232:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.187249:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.187256:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0009 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75497472->75501567] (req 75497472->75501567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.187502:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ece300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503746944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 22371us (22700us total) trans 4294985739 rc 0/0 00000100:00100000:0.0:1789760774.187512:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18443 00000100:00100000:0.0:1789760774.187521:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704274991449 time 704289721846 state 0x2 ref 0 00000100:00100000:3.0:1789760774.197948:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.198050:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747072 00000100:00100000:1.0:1789760774.198059:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747072 00000100:00100000:1.0:1789760774.198110:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704274991449 time 704289721846 state 0x0 ref 1 00000100:00100000:1.0:1789760774.198125:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704289982262 time 704289721846 state 0x1 ref 1 00000100:00100000:1.0:1789760774.198160:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704289982262 time 704322766842 state 0x0 ref 1 00000100:00100000:1.0:1789760774.198173:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503747072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.198217:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75501568, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24020 00010000:00010000:1.0:1789760774.198227:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.198243:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.198247:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.198254:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.198268:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.206431:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75501568 -> 75505664 00002000:00010000:1.0:1789760774.206439:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147488 -> 147496 00010000:00010000:1.0:1789760774.206452:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.206465:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.206474:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.206481:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.206486:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.206494:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.206511:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.206518:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f0010 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75501568->75505663] (req 75501568->75505663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.206594:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503747072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8419us (8652us total) trans 4294985740 rc 0/0 00000100:00100000:1.0:1789760774.206603:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18444 00000100:00100000:1.0:1789760774.206612:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704289982262 time 704322766842 state 0x2 ref 0 00000100:00100000:0.0:1789760774.210728:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747200 00000100:00100000:0.0:1789760774.210748:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747200 00000100:00100000:0.0:1789760774.210803:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704289982262 time 704322766842 state 0x0 ref 1 00000100:00100000:0.0:1789760774.210818:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704323027258 time 704322766842 state 0x1 ref 1 00000100:00100000:0.0:1789760774.210853:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704323027258 time 704335459211 state 0x0 ref 1 00000100:00100000:0.0:1789760774.210866:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503747200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.210912:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75505664, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24028 00010000:00010000:0.0:1789760774.210931:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.210946:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.210951:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.210957:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.210970:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.220777:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75505664 -> 75509760 00002000:00010000:0.0:1789760774.220784:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147496 -> 147504 00010000:00010000:0.0:1789760774.220798:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.220810:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.220824:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.220831:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.220836:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.220845:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.220864:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.220872:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0017 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75505664->75509759] (req 75505664->75509759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.221064:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503747200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10195us (10440us total) trans 4294985741 rc 0/0 00000100:00100000:0.0:1789760774.221074:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18445 00000100:00100000:0.0:1789760774.221084:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704323027258 time 704335459211 state 0x2 ref 0 00000100:00100000:0.0:1789760774.235385:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.235835:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747328 00000100:00100000:1.0:1789760774.235846:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747328 00000100:00100000:1.0:1789760774.235905:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704323027258 time 704335459211 state 0x0 ref 1 00000100:00100000:1.0:1789760774.235921:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704335719627 time 704335459211 state 0x1 ref 1 00000100:00100000:1.0:1789760774.235954:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704335719627 time 704360559716 state 0x0 ref 1 00000100:00100000:1.0:1789760774.235967:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503747328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.236016:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75509760, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24030 00010000:00010000:1.0:1789760774.236024:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.236039:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.236043:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.236049:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.236063:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.242246:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985742 is committed 00002000:00010000:0.0:1789760774.242758:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75509760 -> 75513856 00002000:00010000:0.0:1789760774.242764:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147504 -> 147512 00010000:00010000:0.0:1789760774.242777:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.242790:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.242799:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.242806:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.242811:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.242821:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.242843:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.242851:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f001e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75509760->75513855] (req 75509760->75513855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.242983:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503747328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7013us (7601us total) trans 4294985742 rc 0/0 00000100:00100000:0.0:1789760774.242994:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18446 00000100:00100000:0.0:1789760774.243003:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704335719627 time 704360559716 state 0x2 ref 0 00000100:00100000:3.0:1789760774.248645:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.248751:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747456 00000100:00100000:1.0:1789760774.248761:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747456 00000100:00100000:1.0:1789760774.248818:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704335719627 time 704360559716 state 0x0 ref 1 00000100:00100000:1.0:1789760774.248834:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704360820132 time 704360559716 state 0x1 ref 1 00000100:00100000:1.0:1789760774.248866:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704360820132 time 704373472204 state 0x0 ref 1 00000100:00100000:1.0:1789760774.248879:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503747456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.248925:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75513856, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24038 00010000:00010000:1.0:1789760774.248935:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.248950:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.248955:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.248962:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.248976:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.257049:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985743 is committed 00002000:00010000:1.0:1789760774.264458:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75513856 -> 75517952 00002000:00010000:1.0:1789760774.264468:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147512 -> 147520 00010000:00010000:1.0:1789760774.264485:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.264498:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.264507:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.264515:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.264521:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.264540:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.264560:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.264568:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0025 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75513856->75517951] (req 75513856->75517951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.264850:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503747456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15967us (16220us total) trans 4294985743 rc 0/0 00000100:00100000:1.0:1789760774.265003:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18447 00000100:00100000:1.0:1789760774.265021:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704360820132 time 704373472204 state 0x2 ref 0 00000100:00100000:0.0:1789760774.267875:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747584 00000100:00100000:0.0:1789760774.267885:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747584 00000100:00100000:0.0:1789760774.267940:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704360820132 time 704373472204 state 0x0 ref 1 00000100:00100000:0.0:1789760774.267954:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704373732620 time 704373472204 state 0x1 ref 1 00000100:00100000:0.0:1789760774.267981:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704373732620 time 704392587921 state 0x0 ref 1 00000100:00100000:0.0:1789760774.267994:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503747584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.268039:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75517952, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24040 00010000:00010000:0.0:1789760774.268049:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.268064:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.268069:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.268075:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.268088:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.273828:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985744 is committed 00002000:00010000:0.0:1789760774.274777:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75517952 -> 75522048 00002000:00010000:0.0:1789760774.274784:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147520 -> 147528 00010000:00010000:0.0:1789760774.274798:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.274811:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.274825:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.274833:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.274838:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.274847:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.274867:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.274874:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f002c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75517952->75522047] (req 75517952->75522047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.275144:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503747584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7147us (7364us total) trans 4294985744 rc 0/0 00000100:00100000:0.0:1789760774.275155:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18448 00000100:00100000:0.0:1789760774.275165:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704373732620 time 704392587921 state 0x2 ref 0 00000100:00100000:0.0:1789760774.284973:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760774.285041:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747712 00000100:00100000:3.0:1789760774.285052:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747712 00000100:00100000:3.0:1789760774.285106:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704373732620 time 704392587921 state 0x0 ref 1 00000100:00100000:3.0:1789760774.285120:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704392848337 time 704392587921 state 0x1 ref 1 00000100:00100000:3.0:1789760774.285217:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704392848337 time 704409822574 state 0x0 ref 1 00000100:00100000:3.0:1789760774.285230:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503747712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760774.285276:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75522048, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24048 00010000:00010000:3.0:1789760774.285286:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.285301:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760774.285305:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.285311:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.285323:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.299203:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75522048 -> 75526144 00002000:00010000:0.0:1789760774.299213:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147528 -> 147536 00010000:00010000:0.0:1789760774.299227:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.299241:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.299250:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.299258:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.299314:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.299325:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.299480:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.299490:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0033 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75522048->75526143] (req 75522048->75526143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.300743:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503747712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15437us (15701us total) trans 4294985745 rc 0/0 00000100:00100000:1.0:1789760774.300756:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18449 00000100:00100000:1.0:1789760774.300767:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704392848337 time 704409822574 state 0x2 ref 0 00000100:00100000:3.0:1789760774.305855:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.306122:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747840 00000100:00100000:1.0:1789760774.306133:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747840 00000100:00100000:1.0:1789760774.306187:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704392848337 time 704409822574 state 0x0 ref 1 00000100:00100000:1.0:1789760774.306202:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704410082990 time 704409822574 state 0x1 ref 1 00000100:00100000:1.0:1789760774.306277:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704410082990 time 704430882945 state 0x0 ref 1 00000100:00100000:1.0:1789760774.306290:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503747840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.306337:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75526144, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24050 00010000:00010000:1.0:1789760774.306355:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.306370:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.306375:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.306381:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.306436:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.312304:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75526144 -> 75530240 00002000:00010000:1.0:1789760774.312309:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147536 -> 147544 00010000:00010000:1.0:1789760774.312323:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.312334:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.312344:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.312594:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.312599:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.312608:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.312636:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.312644:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f003a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75526144->75530239] (req 75526144->75530239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.313867:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503747840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7568us (8013us total) trans 4294985746 rc 0/0 00000100:00100000:1.0:1789760774.313881:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18450 00000100:00100000:1.0:1789760774.313891:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704410082990 time 704430882945 state 0x2 ref 0 00000100:00100000:1.0:1789760774.317096:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503747968 00000100:00100000:1.0:1789760774.317106:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503747968 00000100:00100000:1.0:1789760774.317162:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704410082990 time 704430882945 state 0x0 ref 1 00000100:00100000:1.0:1789760774.317176:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704431143361 time 704430882945 state 0x1 ref 1 00000100:00100000:1.0:1789760774.317206:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704431143361 time 704441813353 state 0x0 ref 1 00000100:00100000:1.0:1789760774.317220:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503747968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.317275:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75530240, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24058 00010000:00010000:1.0:1789760774.317283:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.317296:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.317300:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.317306:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.317318:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.324663:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75530240 -> 75534336 00002000:00010000:1.0:1789760774.324671:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147544 -> 147552 00010000:00010000:1.0:1789760774.324685:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.324699:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.324708:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.324716:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.324722:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.324731:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.324749:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.324757:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0041 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75530240->75534335] (req 75530240->75534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.324852:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503747968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7628us (7835us total) trans 4294985747 rc 0/0 00000100:00100000:1.0:1789760774.324863:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18451 00000100:00100000:1.0:1789760774.324872:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704431143361 time 704441813353 state 0x2 ref 0 00000100:00100000:3.0:1789760774.330115:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.330900:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503748096 00000100:00100000:0.0:1789760774.330912:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503748096 00000100:00100000:0.0:1789760774.330968:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704431143361 time 704441813353 state 0x0 ref 1 00000100:00100000:0.0:1789760774.330984:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704442073769 time 704441813353 state 0x1 ref 1 00000100:00100000:0.0:1789760774.331064:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704442073769 time 704455669343 state 0x0 ref 1 00000100:00100000:0.0:1789760774.331080:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503748096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.331129:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75534336, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24060 00010000:00010000:0.0:1789760774.331139:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.331155:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.331160:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.331166:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.331179:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.336642:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75534336 -> 75538432 00002000:00010000:0.0:1789760774.336648:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147552 -> 147560 00010000:00010000:0.0:1789760774.336777:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336795:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336807:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336814:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.336822:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336832:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336854:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336862:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0048 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75534336->75538431] (req 75534336->75538431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.336969:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503748096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5885us (6856us total) trans 4294985748 rc 0/0 00000100:00100000:0.0:1789760774.336980:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18452 00000100:00100000:0.0:1789760774.336990:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704442073769 time 704455669343 state 0x2 ref 0 00000100:00100000:3.0:1789760774.340801:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:1.0:1789760774.356085:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75538432 -> 75542528 00002000:00010000:1.0:1789760774.356095:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147560 -> 147568 00010000:00010000:1.0:1789760774.356110:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f004f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75538432->75542527] (req 75538432->75542527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.356122:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f004f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75538432->75542527] (req 75538432->75542527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.356130:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f004f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75538432->75542527] (req 75538432->75542527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.356154:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.356158:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f004f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75538432->75542527] (req 75538432->75542527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.356171:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f004f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75538432->75542527] (req 75538432->75542527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.356190:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f004f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75538432->75542527] (req 75538432->75542527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.356198:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f004f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75538432->75542527] (req 75538432->75542527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.356294:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503748224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15275us (15499us total) trans 4294985749 rc 0/0 00000100:00100000:1.0:1789760774.356304:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18453 00000100:00100000:1.0:1789760774.356314:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704455929759 time 704465607847 state 0x2 ref 0 00000100:00100000:1.0:1789760774.364220:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503748352 00000100:00100000:1.0:1789760774.364235:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503748352 00000100:00100000:1.0:1789760774.364289:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704455929759 time 704465607847 state 0x0 ref 1 00000100:00100000:1.0:1789760774.364304:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704465868263 time 704465607847 state 0x1 ref 1 00000100:00100000:1.0:1789760774.364324:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704465868263 time 704488931066 state 0x0 ref 1 00000100:00100000:1.0:1789760774.364336:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503748352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.364416:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75542528, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24070 00010000:00010000:1.0:1789760774.364426:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.364440:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.364444:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.364449:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.364462:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.370322:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75542528 -> 75546624 00002000:00010000:1.0:1789760774.370328:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147568 -> 147576 00010000:00010000:1.0:1789760774.370341:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.370364:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.370373:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.370381:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.370387:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.370430:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.370448:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.370457:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0056 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75542528->75546623] (req 75542528->75546623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.370563:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503748352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6222us (6409us total) trans 4294985750 rc 0/0 00000100:00100000:1.0:1789760774.370572:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18454 00000100:00100000:1.0:1789760774.370583:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704465868263 time 704488931066 state 0x2 ref 0 00000100:00100000:3.0:1789760774.375313:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.375538:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503748480 00000100:00100000:1.0:1789760774.375547:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503748480 00000100:00100000:1.0:1789760774.375596:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704465868263 time 704488931066 state 0x0 ref 1 00000100:00100000:1.0:1789760774.375610:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704489191482 time 704488931066 state 0x1 ref 1 00000100:00100000:1.0:1789760774.375645:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704489191482 time 704500251754 state 0x0 ref 1 00000100:00100000:1.0:1789760774.375655:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503748480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760774.375979:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75546624, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24078 00010000:00010000:3.0:1789760774.375989:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.376004:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760774.376008:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.376014:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.376027:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.383708:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75546624 -> 75550720 00002000:00010000:3.0:1789760774.383717:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147576 -> 147584 00010000:00010000:3.0:1789760774.383789:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383803:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383881:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383890:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760774.383895:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383905:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383923:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383930:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f005d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75546624->75550719] (req 75546624->75550719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.384028:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503748480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8369us (8719us total) trans 4294985751 rc 0/0 00000100:00100000:3.0:1789760774.384039:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18455 00000100:00100000:3.0:1789760774.384049:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704489191482 time 704500251754 state 0x2 ref 0 00000100:00100000:3.0:1789760774.391230:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.391666:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503748608 00000100:00100000:0.0:1789760774.391676:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503748608 00000100:00100000:0.0:1789760774.391797:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704489191482 time 704500251754 state 0x0 ref 1 00000100:00100000:0.0:1789760774.391812:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704500512170 time 704500251754 state 0x1 ref 1 00000100:00100000:0.0:1789760774.391846:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704500512170 time 704516452559 state 0x0 ref 1 00000100:00100000:0.0:1789760774.391860:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503748608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.391973:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75550720, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24080 00010000:00010000:0.0:1789760774.391983:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.391998:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.392002:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.392008:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.392020:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.398245:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985752 is committed 00002000:00010000:0.0:1789760774.398463:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75550720 -> 75554816 00002000:00010000:0.0:1789760774.398468:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147584 -> 147592 00010000:00010000:0.0:1789760774.398482:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.398497:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.398506:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.398514:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.398519:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.398528:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.398549:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.398556:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0064 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75550720->75554815] (req 75550720->75554815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.398697:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503748608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6834us (7478us total) trans 4294985752 rc 0/0 00000100:00100000:0.0:1789760774.398716:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18456 00000100:00100000:0.0:1789760774.398726:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704500512170 time 704516452559 state 0x2 ref 0 00000100:00100000:1.0:1789760774.403373:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503748736 00000100:00100000:1.0:1789760774.403383:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503748736 00000100:00100000:1.0:1789760774.403457:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704500512170 time 704516452559 state 0x0 ref 1 00000100:00100000:1.0:1789760774.403483:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704516712975 time 704516452559 state 0x1 ref 1 00000100:00100000:1.0:1789760774.403515:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704516712975 time 704528122149 state 0x0 ref 1 00000100:00100000:1.0:1789760774.403528:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503748736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.403581:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75554816, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24088 00010000:00010000:1.0:1789760774.403590:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.403605:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.403610:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.403616:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.403641:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.412963:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985753 is committed 00002000:00010000:1.0:1789760774.413070:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75554816 -> 75558912 00002000:00010000:1.0:1789760774.413075:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147592 -> 147600 00010000:00010000:1.0:1789760774.413088:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.413098:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.413112:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.413120:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.413125:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.413133:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.413151:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.413158:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f006b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75554816->75558911] (req 75554816->75558911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.413249:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503748736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9716us (9944us total) trans 4294985753 rc 0/0 00000100:00100000:1.0:1789760774.413260:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18457 00000100:00100000:1.0:1789760774.413272:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704516712975 time 704528122149 state 0x2 ref 0 00000100:00100000:3.0:1789760774.416538:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.416674:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503748864 00000100:00100000:1.0:1789760774.416683:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503748864 00000100:00100000:1.0:1789760774.416744:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704516712975 time 704528122149 state 0x0 ref 1 00000100:00100000:1.0:1789760774.416759:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704528382565 time 704528122149 state 0x1 ref 1 00000100:00100000:1.0:1789760774.416780:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704528382565 time 704541387369 state 0x0 ref 1 00000100:00100000:1.0:1789760774.416791:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503748864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.416849:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75558912, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24090 00010000:00010000:1.0:1789760774.416857:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.416873:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.416877:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.416883:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.416896:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.422541:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985754 is committed 00002000:00010000:0.0:1789760774.426562:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75558912 -> 75563008 00002000:00010000:0.0:1789760774.426571:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147600 -> 147608 00010000:00010000:0.0:1789760774.426585:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.426598:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.426608:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.426615:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.426620:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.426644:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.426664:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.426672:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0072 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75558912->75563007] (req 75558912->75563007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.426784:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503748864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9989us (10250us total) trans 4294985754 rc 0/0 00000100:00100000:0.0:1789760774.426795:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18458 00000100:00100000:0.0:1789760774.426804:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704528382565 time 704541387369 state 0x2 ref 0 00000100:00100000:3.0:1789760774.432703:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.432849:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503748992 00000100:00100000:0.0:1789760774.432858:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503748992 00000100:00100000:0.0:1789760774.432908:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704528382565 time 704541387369 state 0x0 ref 1 00000100:00100000:0.0:1789760774.432931:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704541647785 time 704541387369 state 0x1 ref 1 00000100:00100000:0.0:1789760774.432960:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704541647785 time 704557567583 state 0x0 ref 1 00000100:00100000:0.0:1789760774.432972:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503748992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.433016:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75563008, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24098 00010000:00010000:0.0:1789760774.433024:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.433039:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.433043:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.433049:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.433061:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.440141:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985755 is committed 00002000:00010000:0.0:1789760774.440256:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75563008 -> 75567104 00002000:00010000:0.0:1789760774.440262:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147608 -> 147616 00010000:00010000:0.0:1789760774.440276:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.440289:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.440297:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.440304:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.440309:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.440317:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.440340:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.440358:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0079 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75563008->75567103] (req 75563008->75567103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.440492:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503748992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7516us (7801us total) trans 4294985755 rc 0/0 00000100:00100000:0.0:1789760774.440503:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18459 00000100:00100000:0.0:1789760774.440513:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704541647785 time 704557567583 state 0x2 ref 0 00000100:00100000:0.0:1789760774.447673:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503749120 00000100:00100000:0.0:1789760774.447683:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503749120 00000100:00100000:0.0:1789760774.447738:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704541647785 time 704557567583 state 0x0 ref 1 00000100:00100000:0.0:1789760774.447753:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704557827999 time 704557567583 state 0x1 ref 1 00000100:00100000:0.0:1789760774.447784:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704557827999 time 704572391797 state 0x0 ref 1 00000100:00100000:0.0:1789760774.447794:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503749120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.447840:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75567104, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240a0 00010000:00010000:0.0:1789760774.447849:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.447863:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.447868:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.447874:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.447888:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.458501:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985756 is committed 00002000:00010000:0.0:1789760774.459074:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75567104 -> 75571200 00002000:00010000:0.0:1789760774.459081:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147616 -> 147624 00010000:00010000:0.0:1789760774.459094:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459107:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459115:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459122:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.459126:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459204:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459223:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459231:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0080 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75567104->75571199] (req 75567104->75571199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.459336:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503749120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11539us (11780us total) trans 4294985756 rc 0/0 00000100:00100000:0.0:1789760774.459343:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18460 00000100:00100000:0.0:1789760774.459357:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704557827999 time 704572391797 state 0x2 ref 0 00000100:00100000:3.0:1789760774.466865:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.466946:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503749248 00000100:00100000:1.0:1789760774.466958:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503749248 00000100:00100000:1.0:1789760774.467011:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704557827999 time 704572391797 state 0x0 ref 1 00000100:00100000:1.0:1789760774.467026:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704572652213 time 704572391797 state 0x1 ref 1 00000100:00100000:1.0:1789760774.467059:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704572652213 time 704591665002 state 0x0 ref 1 00000100:00100000:1.0:1789760774.467072:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.467118:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75571200, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240a8 00010000:00010000:1.0:1789760774.467128:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.467143:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.467147:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.467153:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.467167:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.473049:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985757 is committed 00002000:00010000:1.0:1789760774.473139:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75571200 -> 75575296 00002000:00010000:1.0:1789760774.473143:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147624 -> 147632 00010000:00010000:1.0:1789760774.473153:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.473161:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.473167:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.473171:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.473174:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.473180:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.473195:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.473200:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0087 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75571200->75575295] (req 75571200->75575295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.473312:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6238us (6452us total) trans 4294985757 rc 0/0 00000100:00100000:1.0:1789760774.473319:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18461 00000100:00100000:1.0:1789760774.473328:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704572652213 time 704591665002 state 0x2 ref 0 00000100:00100000:3.0:1789760774.477276:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760774.486146:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985758 is committed 00000100:00100000:0.0:1789760774.490853:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503749504 00000100:00100000:0.0:1789760774.490861:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503749504 00000100:00100000:0.0:1789760774.491000:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704591925418 time 704602276824 state 0x0 ref 1 00000100:00100000:0.0:1789760774.491015:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704602537240 time 704602276824 state 0x1 ref 1 00000100:00100000:0.0:1789760774.491043:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704602537240 time 704615650340 state 0x0 ref 1 00000100:00100000:0.0:1789760774.491055:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.491105:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75579392, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240b8 00010000:00010000:0.0:1789760774.491113:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.491129:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.491133:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.491139:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.491157:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.496506:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985759 is committed 00002000:00010000:0.0:1789760774.496682:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75579392 -> 75583488 00002000:00010000:0.0:1789760774.496687:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147640 -> 147648 00010000:00010000:0.0:1789760774.496699:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.496718:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.496726:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.496734:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.496738:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.496747:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.496766:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.496774:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0095 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75579392->75583487] (req 75579392->75583487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.496909:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5851us (6217us total) trans 4294985759 rc 0/0 00000100:00100000:0.0:1789760774.496919:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18463 00000100:00100000:0.0:1789760774.496928:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704602537240 time 704615650340 state 0x2 ref 0 00000100:00100000:3.0:1789760774.503598:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.503714:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503749632 00000100:00100000:0.0:1789760774.503724:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503749632 00000100:00100000:0.0:1789760774.503779:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704602537240 time 704615650340 state 0x0 ref 1 00000100:00100000:0.0:1789760774.503794:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704615910756 time 704615650340 state 0x1 ref 1 00000100:00100000:0.0:1789760774.503825:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704615910756 time 704628432050 state 0x0 ref 1 00000100:00100000:0.0:1789760774.503837:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.503884:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75583488, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240c0 00010000:00010000:0.0:1789760774.503894:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.503909:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.503914:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.503920:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.503932:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.508569:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985760 is committed 00002000:00010000:0.0:1789760774.508695:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75583488 -> 75587584 00002000:00010000:0.0:1789760774.508710:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147648 -> 147656 00010000:00010000:0.0:1789760774.508724:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.508736:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.508745:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.508753:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.508757:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.508766:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.508786:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.508794:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f009c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75583488->75587583] (req 75583488->75587583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.508889:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5049us (5296us total) trans 4294985760 rc 0/0 00000100:00100000:0.0:1789760774.508900:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18464 00000100:00100000:0.0:1789760774.508910:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704615910756 time 704628432050 state 0x2 ref 0 00000100:00100000:3.0:1789760774.512329:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.512525:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503749760 00000100:00100000:1.0:1789760774.512535:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503749760 00000100:00100000:1.0:1789760774.512586:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704615910756 time 704628432050 state 0x0 ref 1 00000100:00100000:1.0:1789760774.512600:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704628692466 time 704628432050 state 0x1 ref 1 00000100:00100000:1.0:1789760774.512676:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704628692466 time 704637281163 state 0x0 ref 1 00000100:00100000:1.0:1789760774.512689:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503749760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.512733:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75587584, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240c8 00010000:00010000:1.0:1789760774.512742:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.512758:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.512762:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.512769:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.512782:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.517132:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985761 is committed 00002000:00010000:1.0:1789760774.518045:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75587584 -> 75591680 00002000:00010000:1.0:1789760774.518061:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147656 -> 147664 00010000:00010000:1.0:1789760774.518077:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.518091:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.518100:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.518108:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.518113:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.518122:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.518152:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.518160:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75587584->75591679] (req 75587584->75591679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.518250:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503749760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5559us (5926us total) trans 4294985761 rc 0/0 00000100:00100000:1.0:1789760774.518261:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18465 00000100:00100000:1.0:1789760774.518270:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704628692466 time 704637281163 state 0x2 ref 0 00000100:00100000:0.0:1789760774.523877:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503749888 00000100:00100000:0.0:1789760774.523887:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503749888 00000100:00100000:0.0:1789760774.523941:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704628692466 time 704637281163 state 0x0 ref 1 00000100:00100000:0.0:1789760774.523956:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704637541579 time 704637281163 state 0x1 ref 1 00000100:00100000:0.0:1789760774.523987:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704637541579 time 704648593548 state 0x0 ref 1 00000100:00100000:0.0:1789760774.523998:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.524045:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75591680, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240d0 00010000:00010000:0.0:1789760774.524054:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.524069:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.524073:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.524079:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.524090:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.528079:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985762 is committed 00002000:00010000:0.0:1789760774.528194:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75591680 -> 75595776 00002000:00010000:0.0:1789760774.528199:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147664 -> 147672 00010000:00010000:0.0:1789760774.528212:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.528225:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.528233:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.528240:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.528244:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.528251:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.528268:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.528274:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75591680->75595775] (req 75591680->75595775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.528430:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503749888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4391us (4610us total) trans 4294985762 rc 0/0 00000100:00100000:0.0:1789760774.528439:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18466 00000100:00100000:0.0:1789760774.528448:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704637541579 time 704648593548 state 0x2 ref 0 00000100:00100000:3.0:1789760774.531609:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.531695:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503750016 00000100:00100000:0.0:1789760774.531714:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503750016 00000100:00100000:0.0:1789760774.531766:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704637541579 time 704648593548 state 0x0 ref 1 00000100:00100000:0.0:1789760774.531780:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704648853964 time 704648593548 state 0x1 ref 1 00000100:00100000:0.0:1789760774.531812:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704648853964 time 704656418224 state 0x0 ref 1 00000100:00100000:0.0:1789760774.531824:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503750016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.531868:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75595776, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240d8 00010000:00010000:0.0:1789760774.531877:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.531892:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.531897:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.531902:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.531915:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.535829:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985763 is committed 00002000:00010000:0.0:1789760774.536421:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75595776 -> 75599872 00002000:00010000:0.0:1789760774.536426:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147672 -> 147680 00010000:00010000:0.0:1789760774.536441:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.536457:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.536465:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.536472:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.536476:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.536484:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.536502:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.536510:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f00b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75595776->75599871] (req 75595776->75599871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.536605:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503750016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4778us (4999us total) trans 4294985763 rc 0/0 00000100:00100000:0.0:1789760774.536615:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18467 00000100:00100000:0.0:1789760774.536633:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704648853964 time 704656418224 state 0x2 ref 0 00000100:00100000:3.0:1789760774.541077:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.541278:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503750144 00000100:00100000:1.0:1789760774.541288:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503750144 00000100:00100000:1.0:1789760774.541342:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704648853964 time 704656418224 state 0x0 ref 1 00000100:00100000:1.0:1789760774.541451:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704656678640 time 704656418224 state 0x1 ref 1 00000100:00100000:1.0:1789760774.541480:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704656678640 time 704666086449 state 0x0 ref 1 00000100:00100000:1.0:1789760774.541492:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503750144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.541538:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75599872, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240e0 00010000:00010000:1.0:1789760774.541547:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.541561:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.541567:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.541573:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.541587:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.545341:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985764 is committed 00002000:00010000:1.0:1789760774.545885:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75599872 -> 75603968 00002000:00010000:1.0:1789760774.545891:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147680 -> 147688 00010000:00010000:1.0:1789760774.545904:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.545917:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.545989:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.545998:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.546003:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.546013:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.546033:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.546047:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75599872->75603967] (req 75599872->75603967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.546245:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503750144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4750us (5178us total) trans 4294985764 rc 0/0 00000100:00100000:1.0:1789760774.546255:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18468 00000100:00100000:1.0:1789760774.546265:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704656678640 time 704666086449 state 0x2 ref 0 00000100:00100000:1.0:1789760774.549568:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503750272 00000100:00100000:1.0:1789760774.549577:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503750272 00000100:00100000:1.0:1789760774.549639:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704656678640 time 704666086449 state 0x0 ref 1 00000100:00100000:1.0:1789760774.549654:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704666346865 time 704666086449 state 0x1 ref 1 00000100:00100000:1.0:1789760774.549683:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704666346865 time 704674290104 state 0x0 ref 1 00000100:00100000:1.0:1789760774.549696:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.549740:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75603968, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240e8 00010000:00010000:1.0:1789760774.549749:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.549764:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.549768:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.549775:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.549793:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.555043:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985765 is committed 00002000:00010000:1.0:1789760774.555217:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75603968 -> 75608064 00002000:00010000:1.0:1789760774.555223:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147688 -> 147696 00010000:00010000:1.0:1789760774.555235:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.555249:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.555258:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.555265:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.555270:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.555278:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.555296:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.555303:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75603968->75608063] (req 75603968->75608063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.555458:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5759us (5979us total) trans 4294985765 rc 0/0 00000100:00100000:1.0:1789760774.555479:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18469 00000100:00100000:1.0:1789760774.555489:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704666346865 time 704674290104 state 0x2 ref 0 00000100:00100000:3.0:1789760774.561636:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760774.579446:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985766 is committed 00000100:00100000:3.0:1789760774.588955:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.589026:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503750528 00000100:00100000:0.0:1789760774.589036:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503750528 00000100:00100000:0.0:1789760774.589089:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704674550520 time 704686570862 state 0x0 ref 1 00000100:00100000:0.0:1789760774.589103:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704686831278 time 704686570862 state 0x1 ref 1 00000100:00100000:1.0:1789760774.589233:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704686831278 time 704713837123 state 0x0 ref 1 00000100:00100000:1.0:1789760774.589247:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.589294:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75612160, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x240f8 00010000:00010000:1.0:1789760774.589303:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.589319:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.589323:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.589330:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.589343:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.597852:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75612160 -> 75616256 00002000:00010000:1.0:1789760774.597857:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147704 -> 147712 00010000:00010000:1.0:1789760774.597870:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.597883:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.597891:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.597898:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.597903:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.597912:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.597932:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.597940:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75612160->75616255] (req 75612160->75616255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.598140:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8890us (9192us total) trans 4294985767 rc 0/0 00000100:00100000:1.0:1789760774.598150:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18471 00000100:00100000:1.0:1789760774.598160:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704686831278 time 704713837123 state 0x2 ref 0 00000100:00100000:0.0:1789760774.609319:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.609433:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503750656 00000100:00100000:1.0:1789760774.609443:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503750656 00000100:00100000:1.0:1789760774.609492:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704686831278 time 704713837123 state 0x0 ref 1 00000100:00100000:1.0:1789760774.609507:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704714097539 time 704713837123 state 0x1 ref 1 00000100:00100000:1.0:1789760774.609536:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704714097539 time 704734142639 state 0x0 ref 1 00000100:00100000:1.0:1789760774.609547:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.609589:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75616256, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24100 00010000:00010000:1.0:1789760774.609598:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.609612:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.609617:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.609634:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.609648:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.614976:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75616256 -> 75620352 00002000:00010000:1.0:1789760774.614982:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147712 -> 147720 00010000:00010000:1.0:1789760774.614995:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.615009:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.615022:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.615029:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.615033:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.615042:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.615072:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.615081:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75616256->75620351] (req 75616256->75620351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.615226:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5676us (5912us total) trans 4294985768 rc 0/0 00000100:00100000:1.0:1789760774.615236:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18472 00000100:00100000:1.0:1789760774.615246:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704714097539 time 704734142639 state 0x2 ref 0 00000100:00100000:3.0:1789760774.618977:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.619066:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503750784 00000100:00100000:0.0:1789760774.619077:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503750784 00000100:00100000:0.0:1789760774.619131:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704714097539 time 704734142639 state 0x0 ref 1 00000100:00100000:0.0:1789760774.619146:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704734403055 time 704734142639 state 0x1 ref 1 00000100:00100000:0.0:1789760774.619176:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704734403055 time 704743782533 state 0x0 ref 1 00000100:00100000:0.0:1789760774.619194:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.619238:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75620352, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24108 00010000:00010000:0.0:1789760774.619248:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.619264:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.619268:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.619274:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.619286:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.626417:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985769 is committed 00002000:00010000:0.0:1789760774.626558:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75620352 -> 75624448 00002000:00010000:0.0:1789760774.626564:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147720 -> 147728 00010000:00010000:0.0:1789760774.626578:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.626590:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.626600:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.626607:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.626612:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.626632:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.626653:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.626720:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75620352->75624447] (req 75620352->75624447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.627643:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8444us (8667us total) trans 4294985769 rc 0/0 00000100:00100000:0.0:1789760774.627656:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18473 00000100:00100000:0.0:1789760774.627675:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704734403055 time 704743782533 state 0x2 ref 0 00000100:00100000:0.0:1789760774.632080:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503750912 00000100:00100000:0.0:1789760774.632090:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503750912 00000100:00100000:0.0:1789760774.632147:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704734403055 time 704743782533 state 0x0 ref 1 00000100:00100000:0.0:1789760774.632162:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704744042949 time 704743782533 state 0x1 ref 1 00000100:00100000:0.0:1789760774.632197:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704744042949 time 704756803808 state 0x0 ref 1 00000100:00100000:0.0:1789760774.632210:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.632256:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75624448, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24110 00010000:00010000:0.0:1789760774.632265:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.632280:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.632285:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.632291:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.632305:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.637315:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985770 is committed 00002000:00010000:0.0:1789760774.637824:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75624448 -> 75628544 00002000:00010000:0.0:1789760774.637830:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147728 -> 147736 00010000:00010000:0.0:1789760774.637843:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.637863:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.637872:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.637879:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.637885:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.637894:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.638227:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.638235:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75624448->75628543] (req 75624448->75628543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.638433:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503750912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6175us (6388us total) trans 4294985770 rc 0/0 00000100:00100000:0.0:1789760774.638445:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18474 00000100:00100000:0.0:1789760774.638454:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704744042949 time 704756803808 state 0x2 ref 0 00000100:00100000:1.0:1789760774.644759:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503751040 00000100:00100000:1.0:1789760774.644768:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503751040 00000100:00100000:1.0:1789760774.644818:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704744042949 time 704756803808 state 0x0 ref 1 00000100:00100000:1.0:1789760774.644832:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704757064224 time 704756803808 state 0x1 ref 1 00000100:00100000:1.0:1789760774.644883:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704757064224 time 704769489899 state 0x0 ref 1 00000100:00100000:1.0:1789760774.644895:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.644947:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75628544, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24118 00010000:00010000:1.0:1789760774.644956:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.644972:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.644976:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.644982:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.644996:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.649564:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75628544 -> 75632640 00002000:00010000:1.0:1789760774.649569:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147736 -> 147744 00010000:00010000:1.0:1789760774.649582:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.649594:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.649609:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.649617:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.649630:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.649640:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.649777:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.649788:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f00e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75628544->75632639] (req 75628544->75632639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.649874:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4975us (5317us total) trans 4294985771 rc 0/0 00000100:00100000:1.0:1789760774.649883:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18475 00000100:00100000:1.0:1789760774.649893:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704757064224 time 704769489899 state 0x2 ref 0 00000100:00100000:3.0:1789760774.654582:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.655144:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503751168 00000100:00100000:1.0:1789760774.655153:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503751168 00000100:00100000:1.0:1789760774.655211:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704757064224 time 704769489899 state 0x0 ref 1 00000100:00100000:1.0:1789760774.655226:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704769750315 time 704769489899 state 0x1 ref 1 00000100:00100000:1.0:1789760774.655303:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704769750315 time 704779908383 state 0x0 ref 1 00000100:00100000:1.0:1789760774.655316:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.655371:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75632640, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24120 00010000:00010000:1.0:1789760774.655381:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.655440:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.655445:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.655452:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.655467:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.660633:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75632640 -> 75636736 00002000:00010000:0.0:1789760774.660639:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147744 -> 147752 00010000:00010000:0.0:1789760774.660652:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.660736:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.660745:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.660752:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.660756:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.660765:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.660785:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.660793:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f00f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75632640->75636735] (req 75632640->75636735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.661024:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5705us (6446us total) trans 4294985772 rc 0/0 00000100:00100000:0.0:1789760774.661035:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18476 00000100:00100000:0.0:1789760774.661044:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704769750315 time 704779908383 state 0x2 ref 0 00000100:00100000:0.0:1789760774.664924:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503751296 00000100:00100000:0.0:1789760774.664934:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503751296 00000100:00100000:0.0:1789760774.664988:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704769750315 time 704779908383 state 0x0 ref 1 00000100:00100000:0.0:1789760774.665003:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704780168799 time 704779908383 state 0x1 ref 1 00000100:00100000:0.0:1789760774.665034:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704780168799 time 704789640477 state 0x0 ref 1 00000100:00100000:0.0:1789760774.665046:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc15c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503751296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.665090:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75636736, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24128 00010000:00010000:0.0:1789760774.665099:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.665115:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.665119:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.665125:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.665138:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.669636:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75636736 -> 75640832 00002000:00010000:0.0:1789760774.669641:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147752 -> 147760 00010000:00010000:0.0:1789760774.669662:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.669681:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.669691:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.669699:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.669704:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.669713:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.669732:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.669741:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f00f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75636736->75640831] (req 75636736->75640831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.669837:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc15c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503751296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4789us (4999us total) trans 4294985773 rc 0/0 00000100:00100000:0.0:1789760774.669848:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18477 00000100:00100000:0.0:1789760774.669858:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704780168799 time 704789640477 state 0x2 ref 0 00000100:00100000:0.0:1789760774.675066:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503751424 00000100:00100000:0.0:1789760774.675077:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503751424 00000100:00100000:0.0:1789760774.675215:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704780168799 time 704789640477 state 0x0 ref 1 00000100:00100000:0.0:1789760774.675229:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704789900893 time 704789640477 state 0x1 ref 1 00000100:00100000:0.0:1789760774.675261:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704789900893 time 704799867514 state 0x0 ref 1 00000100:00100000:0.0:1789760774.675273:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.675318:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75640832, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24130 00010000:00010000:0.0:1789760774.675327:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.675342:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.675348:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.675362:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.675376:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.679819:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985774 is committed 00002000:00010000:0.0:1789760774.679973:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75640832 -> 75644928 00002000:00010000:0.0:1789760774.679979:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147760 -> 147768 00010000:00010000:0.0:1789760774.680003:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.680016:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.680031:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.680038:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.680043:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.680053:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.680072:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.680080:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f00fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75640832->75644927] (req 75640832->75644927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.680177:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4901us (5864us total) trans 4294985774 rc 0/0 00000100:00100000:0.0:1789760774.680188:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18478 00000100:00100000:0.0:1789760774.680197:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704789900893 time 704799867514 state 0x2 ref 0 00010000:00010000:3.0:1789760774.682258:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760774.684767:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.684941:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503751552 00000100:00100000:1.0:1789760774.684952:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503751552 00000100:00100000:1.0:1789760774.685074:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704789900893 time 704799867514 state 0x0 ref 1 00000100:00100000:1.0:1789760774.685090:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704800127930 time 704799867514 state 0x1 ref 1 00000100:00100000:1.0:1789760774.685121:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704800127930 time 704809727200 state 0x0 ref 1 00000100:00100000:1.0:1789760774.685135:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503751552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.685191:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75644928, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24138 00010000:00010000:1.0:1789760774.685201:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.685220:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.685225:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.685231:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.685246:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.691117:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75644928 -> 75649024 00002000:00010000:1.0:1789760774.691125:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147768 -> 147776 00010000:00010000:1.0:1789760774.691139:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.691152:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.691161:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.691167:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.691172:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.691186:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.691206:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.691214:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0105 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75644928->75649023] (req 75644928->75649023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.693226:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503751552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6205us (6591us total) trans 4294985775 rc 0/0 00000100:00100000:1.0:1789760774.693247:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18479 00000100:00100000:1.0:1789760774.693258:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704800127930 time 704809727200 state 0x2 ref 0 00000100:00100000:1.0:1789760774.697347:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.697643:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503751680 00000100:00100000:0.0:1789760774.697654:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503751680 00000100:00100000:0.0:1789760774.697716:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704800127930 time 704809727200 state 0x0 ref 1 00000100:00100000:0.0:1789760774.697732:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704809987616 time 704809727200 state 0x1 ref 1 00000100:00100000:0.0:1789760774.697762:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704809987616 time 704822368950 state 0x0 ref 1 00000100:00100000:0.0:1789760774.697775:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.697825:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75649024, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24140 00010000:00010000:0.0:1789760774.697835:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.697850:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.697854:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.697860:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.697873:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.702181:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75649024 -> 75653120 00002000:00010000:0.0:1789760774.702187:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147776 -> 147784 00010000:00010000:0.0:1789760774.702199:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.702212:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.702222:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.702235:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.702240:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.702248:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.702269:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.702276:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f010c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75649024->75653119] (req 75649024->75653119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.702379:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503751680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4600us (5035us total) trans 4294985776 rc 0/0 00000100:00100000:0.0:1789760774.702411:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18480 00000100:00100000:0.0:1789760774.702422:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704809987616 time 704822368950 state 0x2 ref 0 00000100:00100000:3.0:1789760774.711131:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503751808 00000100:00100000:3.0:1789760774.711140:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503751808 00000100:00100000:3.0:1789760774.711194:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704809987616 time 704822368950 state 0x0 ref 1 00000100:00100000:3.0:1789760774.711209:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704822629366 time 704822368950 state 0x1 ref 1 00000100:00100000:3.0:1789760774.711275:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704822629366 time 704835880992 state 0x0 ref 1 00000100:00100000:3.0:1789760774.711288:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503751808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760774.711337:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75653120, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24148 00010000:00010000:3.0:1789760774.711357:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.711372:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760774.711376:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.711382:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.711415:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.715815:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75653120 -> 75657216 00002000:00010000:0.0:1789760774.715820:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147784 -> 147792 00010000:00010000:0.0:1789760774.715830:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.715906:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.715921:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.715929:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.715933:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.715942:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.715961:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.715969:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0113 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75653120->75657215] (req 75653120->75657215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.716072:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503751808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4782us (5040us total) trans 4294985777 rc 0/0 00000100:00100000:0.0:1789760774.716082:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18481 00000100:00100000:0.0:1789760774.716091:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704822629366 time 704835880992 state 0x2 ref 0 00000100:00100000:3.0:1789760774.718870:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760774.724814:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985778 is committed 00000100:00100000:1.0:1789760774.729814:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760774.734922:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985779 is committed 00000100:00100000:0.0:1789760774.739360:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.739483:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503752192 00000100:00100000:1.0:1789760774.739495:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503752192 00000100:00100000:1.0:1789760774.739555:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704843936101 time 704854705002 state 0x0 ref 1 00000100:00100000:1.0:1789760774.739569:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704854965418 time 704854705002 state 0x1 ref 1 00000100:00100000:1.0:1789760774.739642:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704854965418 time 704864247909 state 0x0 ref 1 00000100:00100000:1.0:1789760774.739655:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503752192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.739711:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75665408, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24160 00010000:00010000:1.0:1789760774.739720:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.739735:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.739739:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.739745:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.739758:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.745280:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985780 is committed 00002000:00010000:1.0:1789760774.745423:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75665408 -> 75669504 00002000:00010000:1.0:1789760774.745428:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147808 -> 147816 00010000:00010000:1.0:1789760774.745442:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.745455:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.745474:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.745483:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.745488:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.745503:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.745522:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.745530:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0128 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75665408->75669503] (req 75665408->75669503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.745823:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503752192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6164us (6464us total) trans 4294985780 rc 0/0 00000100:00100000:1.0:1789760774.745834:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18484 00000100:00100000:1.0:1789760774.745844:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704854965418 time 704864247909 state 0x2 ref 0 00000100:00100000:3.0:1789760774.748919:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.749038:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503752320 00000100:00100000:0.0:1789760774.749048:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503752320 00000100:00100000:0.0:1789760774.749100:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704854965418 time 704864247909 state 0x0 ref 1 00000100:00100000:0.0:1789760774.749119:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704864508325 time 704864247909 state 0x1 ref 1 00000100:00100000:0.0:1789760774.749151:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704864508325 time 704873757198 state 0x0 ref 1 00000100:00100000:0.0:1789760774.749163:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503752320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.749209:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75669504, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24168 00010000:00010000:0.0:1789760774.749219:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.749234:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.749239:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.749245:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.749257:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.755039:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985781 is committed 00002000:00010000:0.0:1789760774.755229:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75669504 -> 75673600 00002000:00010000:0.0:1789760774.755234:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147816 -> 147824 00010000:00010000:0.0:1789760774.755247:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.755260:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.755269:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.755277:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.755282:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.755291:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.755309:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.755316:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f012f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75669504->75673599] (req 75669504->75673599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.755615:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503752320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6441us (6692us total) trans 4294985781 rc 0/0 00000100:00100000:0.0:1789760774.755634:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18485 00000100:00100000:0.0:1789760774.755642:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704864508325 time 704873757198 state 0x2 ref 0 00000100:00100000:0.0:1789760774.760901:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503752448 00000100:00100000:0.0:1789760774.760910:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503752448 00000100:00100000:0.0:1789760774.760963:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704864508325 time 704873757198 state 0x0 ref 1 00000100:00100000:0.0:1789760774.760977:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704874017614 time 704873757198 state 0x1 ref 1 00000100:00100000:0.0:1789760774.760999:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704874017614 time 704885605882 state 0x0 ref 1 00000100:00100000:0.0:1789760774.761014:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503752448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.761059:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75673600, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24170 00010000:00010000:0.0:1789760774.761068:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.761083:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.761088:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.761095:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.761108:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.764884:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985782 is committed 00002000:00010000:0.0:1789760774.765107:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75673600 -> 75677696 00002000:00010000:0.0:1789760774.765113:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147824 -> 147832 00010000:00010000:0.0:1789760774.765127:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.765141:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.765150:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.765157:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.765163:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.765172:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.765194:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.765202:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0136 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75673600->75677695] (req 75673600->75677695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.765334:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503752448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4318us (4616us total) trans 4294985782 rc 0/0 00000100:00100000:0.0:1789760774.765354:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18486 00000100:00100000:0.0:1789760774.765364:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704874017614 time 704885605882 state 0x2 ref 0 00000100:00100000:3.0:1789760774.768824:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503752576 00000100:00100000:3.0:1789760774.768839:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503752576 00000100:00100000:3.0:1789760774.768905:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704874017614 time 704885605882 state 0x0 ref 1 00000100:00100000:3.0:1789760774.768921:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704885866298 time 704885605882 state 0x1 ref 1 00000100:00100000:3.0:1789760774.768947:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704885866298 time 704893553865 state 0x0 ref 1 00000100:00100000:3.0:1789760774.768960:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503752576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760774.769009:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75677696, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24178 00010000:00010000:3.0:1789760774.769018:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.769032:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760774.769037:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.769043:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.769057:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.773149:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985783 is committed 00002000:00010000:3.0:1789760774.773239:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75677696 -> 75681792 00002000:00010000:3.0:1789760774.773245:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147832 -> 147840 00010000:00010000:3.0:1789760774.773256:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.773268:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.773277:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.773284:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760774.773289:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.773304:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.773322:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.773329:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f013d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75677696->75681791] (req 75677696->75681791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.778221:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503752704 00000100:00100000:1.0:1789760774.778232:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503752704 00000100:00100000:1.0:1789760774.778290:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704885866298 time 704893553865 state 0x0 ref 1 00000100:00100000:1.0:1789760774.778305:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704893814281 time 704893553865 state 0x1 ref 1 00000100:00100000:1.0:1789760774.778337:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704893814281 time 704902943180 state 0x0 ref 1 00000100:00100000:1.0:1789760774.778355:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503752704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.778436:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75681792, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x24180 00010000:00010000:1.0:1789760774.778447:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778463:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760774.778467:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.778474:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778488:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.783633:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985784 is committed 00002000:00010000:0.0:1789760774.783858:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75681792 -> 75685888 00002000:00010000:0.0:1789760774.783863:0:15091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:0.0:1789760774.783866:0:15091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00002000:00010000:0.0:1789760774.783868:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147840 -> 147848 00010000:00010000:0.0:1789760774.783882:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.783895:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.783910:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.783917:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.783923:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.783932:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.783951:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.783959:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0144 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75681792->75685887] (req 75681792->75685887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.784176:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503752704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5823us (6056us total) trans 4294985784 rc 0/0 00000100:00100000:0.0:1789760774.784187:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18488 00000100:00100000:0.0:1789760774.784197:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704893814281 time 704902943180 state 0x2 ref 0 00000100:00100000:1.0:1789760774.789865:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760774.793789:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985785 is committed 00000100:00100000:0.0:1789760774.797566:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503752960 00000100:00100000:0.0:1789760774.797577:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503752960 00000100:00100000:0.0:1789760774.797639:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704903203596 time 704914724027 state 0x0 ref 1 00000100:00100000:0.0:1789760774.797778:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704914984443 time 704914724027 state 0x1 ref 1 00000100:00100000:0.0:1789760774.797813:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704914984443 time 704922418470 state 0x0 ref 1 00000100:00100000:0.0:1789760774.797826:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503752960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.797875:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75689984, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24190 00010000:00010000:0.0:1789760774.797884:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.797902:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.797907:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.797914:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.797927:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.802632:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985786 is committed 00002000:00010000:0.0:1789760774.802937:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75689984 -> 75694080 00002000:00010000:0.0:1789760774.802944:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147856 -> 147864 00010000:00010000:0.0:1789760774.802957:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.802977:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.802986:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.802993:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.802998:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.803007:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.803026:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.803034:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0152 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75689984->75694079] (req 75689984->75694079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.803168:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503752960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5339us (5682us total) trans 4294985786 rc 0/0 00000100:00100000:0.0:1789760774.803180:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18490 00000100:00100000:0.0:1789760774.803210:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704914984443 time 704922418470 state 0x2 ref 0 00000100:00100000:0.0:1789760774.808262:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760774.812860:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985787 is committed 00000100:00100000:1.0:1789760774.818874:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760774.823442:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985788 is committed 00000100:00100000:0.0:1789760774.826764:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760774.835872:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985789 is committed 00000100:00100000:0.0:1789760774.845602:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760774.852270:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985790 is committed 00000100:00100000:1.0:1789760774.855232:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.855384:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503753600 00000100:00100000:0.0:1789760774.855421:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503753600 00000100:00100000:0.0:1789760774.855474:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704953962781 time 704970481072 state 0x0 ref 1 00000100:00100000:0.0:1789760774.855489:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704970741488 time 704970481072 state 0x1 ref 1 00000100:00100000:0.0:1789760774.855512:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704970741488 time 704980119751 state 0x0 ref 1 00000100:00100000:0.0:1789760774.855524:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503753600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.855565:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75710464, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x241b8 00010000:00010000:0.0:1789760774.855573:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.855587:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760774.855592:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.855598:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.855610:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.862081:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985791 is committed 00002000:00010000:1.0:1789760774.862250:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75710464 -> 75714560 00002000:00010000:1.0:1789760774.862256:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147896 -> 147904 00010000:00010000:1.0:1789760774.862269:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.862282:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.862290:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.862297:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.862302:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.862310:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.862329:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.862337:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0175 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75710464->75714559] (req 75710464->75714559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.862629:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503753600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7094us (7391us total) trans 4294985791 rc 0/0 00000100:00100000:1.0:1789760774.862639:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18495 00000100:00100000:1.0:1789760774.862648:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704970741488 time 704980119751 state 0x2 ref 0 00000100:00100000:0.0:1789760774.865522:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760774.870000:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75714560 -> 75718656 00002000:00010000:0.0:1789760774.870005:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147904 -> 147912 00010000:00010000:0.0:1789760774.870080:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f017c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75714560->75718655] (req 75714560->75718655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.870095:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f017c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75714560->75718655] (req 75714560->75718655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.870109:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f017c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75714560->75718655] (req 75714560->75718655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.870117:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.870122:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f017c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75714560->75718655] (req 75714560->75718655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.870130:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f017c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75714560->75718655] (req 75714560->75718655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.870148:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f017c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75714560->75718655] (req 75714560->75718655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.870155:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f017c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75714560->75718655] (req 75714560->75718655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.870244:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503753728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4251us (4725us total) trans 4294985792 rc 0/0 00000100:00100000:0.0:1789760774.870254:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18496 00000100:00100000:0.0:1789760774.870264:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704980380167 time 704990584889 state 0x2 ref 0 00000100:00100000:0.0:1789760774.873109:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760774.879941:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985793 is committed 00000100:00100000:1.0:1789760774.887491:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760774.887643:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503753984 00000100:00100000:0.0:1789760774.887652:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503753984 00000100:00100000:0.0:1789760774.887709:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704990845305 time 704997928664 state 0x0 ref 1 00000100:00100000:0.0:1789760774.887726:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704998189080 time 704997928664 state 0x1 ref 1 00000100:00100000:0.0:1789760774.887764:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704998189080 time 705012370089 state 0x0 ref 1 00000100:00100000:0.0:1789760774.887776:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503753984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.887825:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75722752, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x241d0 00010000:00010000:0.0:1789760774.887834:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.887848:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760774.887852:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.887858:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.887872:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.896994:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985794 is committed 00002000:00010000:0.0:1789760774.897380:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75722752 -> 75726848 00002000:00010000:0.0:1789760774.897386:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147920 -> 147928 00010000:00010000:0.0:1789760774.897427:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897441:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897450:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897458:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.897463:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897472:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897492:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897498:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f018a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75722752->75726847] (req 75722752->75726847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.897592:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503753984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9814us (10106us total) trans 4294985794 rc 0/0 00000100:00100000:0.0:1789760774.897602:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18498 00000100:00100000:0.0:1789760774.897613:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704998189080 time 705012370089 state 0x2 ref 0 00000100:00100000:0.0:1789760774.903805:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760774.903949:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503754112 00000100:00100000:1.0:1789760774.903961:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503754112 00000100:00100000:1.0:1789760774.904025:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 704998189080 time 705012370089 state 0x0 ref 1 00000100:00100000:1.0:1789760774.904041:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705012630505 time 705012370089 state 0x1 ref 1 00000100:00100000:1.0:1789760774.904070:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705012630505 time 705028676995 state 0x0 ref 1 00000100:00100000:1.0:1789760774.904087:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.904132:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75726848, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x241d8 00010000:00010000:1.0:1789760774.904141:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.904156:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760774.904161:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.904168:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.904181:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.914322:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75726848 -> 75730944 00002000:00010000:1.0:1789760774.914335:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147928 -> 147936 00010000:00010000:1.0:1789760774.914355:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.914371:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.914381:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.914443:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760774.914450:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.914462:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.914492:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.914500:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0191 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75726848->75730943] (req 75726848->75730943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.914661:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10563us (10851us total) trans 4294985795 rc 0/0 00000100:00100000:1.0:1789760774.914675:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18499 00000100:00100000:1.0:1789760774.916959:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705012630505 time 705028676995 state 0x2 ref 0 00000100:00100000:0.0:1789760774.921917:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503754240 00000100:00100000:0.0:1789760774.921930:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503754240 00000100:00100000:0.0:1789760774.922007:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705012630505 time 705028676995 state 0x0 ref 1 00000100:00100000:0.0:1789760774.922025:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705028937411 time 705028676995 state 0x1 ref 1 00000100:00100000:0.0:1789760774.922060:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705028937411 time 705046667534 state 0x0 ref 1 00000100:00100000:0.0:1789760774.922081:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.922145:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75730944, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x241e0 00010000:00010000:0.0:1789760774.922156:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.922176:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760774.922181:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.922188:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.922204:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.927989:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985796 is committed 00002000:00010000:0.0:1789760774.928103:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75730944 -> 75735040 00002000:00010000:0.0:1789760774.928109:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147936 -> 147944 00010000:00010000:0.0:1789760774.928128:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.928144:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.928154:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.928160:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.928165:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.928173:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.928194:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.928198:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0198 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75730944->75735039] (req 75730944->75735039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.928573:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6486us (6744us total) trans 4294985796 rc 0/0 00000100:00100000:0.0:1789760774.928587:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18500 00000100:00100000:0.0:1789760774.928600:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705028937411 time 705046667534 state 0x2 ref 0 00000100:00100000:0.0:1789760774.933212:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503754368 00000100:00100000:0.0:1789760774.933226:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503754368 00000100:00100000:0.0:1789760774.933301:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705028937411 time 705046667534 state 0x0 ref 1 00000100:00100000:0.0:1789760774.933319:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705046927950 time 705046667534 state 0x1 ref 1 00000100:00100000:1.0:1789760774.935125:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705046927950 time 705059657334 state 0x0 ref 1 00000100:00100000:1.0:1789760774.935164:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760774.935450:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75735040, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x241e8 00010000:00010000:1.0:1789760774.935461:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f019f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.935480:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760774.935486:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.935493:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f019f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75735040->75739135] (req 75735040->75739135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.935510:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f019f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75735040->75739135] (req 75735040->75739135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.940610:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985797 is committed 00000100:00100000:0.0:1789760774.948836:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503754496 00000100:00100000:0.0:1789760774.948857:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503754496 00000100:00100000:0.0:1789760774.948943:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705046927950 time 705059657334 state 0x0 ref 1 00000100:00100000:0.0:1789760774.948964:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705059917750 time 705059657334 state 0x1 ref 1 00000100:00100000:0.0:1789760774.949055:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705059917750 time 705073660243 state 0x0 ref 1 00000100:00100000:0.0:1789760774.949079:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.949145:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75739136, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x241f0 00010000:00010000:0.0:1789760774.949157:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.949178:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760774.949184:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.949191:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.949207:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.955583:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985798 is committed 00002000:00010000:0.0:1789760774.955939:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75739136 -> 75743232 00002000:00010000:0.0:1789760774.955946:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147952 -> 147960 00010000:00010000:0.0:1789760774.955963:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.955978:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.955988:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.956870:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760774.956877:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.956888:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.956910:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.956919:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75739136->75743231] (req 75739136->75743231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.957025:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7946us (9848us total) trans 4294985798 rc 0/0 00000100:00100000:0.0:1789760774.957035:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18502 00000100:00100000:0.0:1789760774.957046:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705059917750 time 705073660243 state 0x2 ref 0 00000100:00100000:0.0:1789760774.963134:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760774.974201:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985799 is committed 00000100:00100000:0.0:1789760774.982079:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503754752 00000100:00100000:0.0:1789760774.982094:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503754752 00000100:00100000:0.0:1789760774.982167:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705073920659 time 705088218339 state 0x0 ref 1 00000100:00100000:0.0:1789760774.982185:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705088478755 time 705088218339 state 0x1 ref 1 00000100:00100000:0.0:1789760774.982222:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705088478755 time 705106828446 state 0x0 ref 1 00000100:00100000:0.0:1789760774.982237:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503754752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760774.982314:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75747328, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24200 00010000:00010000:0.0:1789760774.982324:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f01b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.982343:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760774.982348:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.982356:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f01b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75747328->75751423] (req 75747328->75751423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.982372:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f01b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75747328->75751423] (req 75747328->75751423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.992468:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985800 is committed 00000100:00100000:0.0:1789760775.001700:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760775.001968:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503754880 00000100:00100000:1.0:1789760775.001979:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503754880 00000100:00100000:1.0:1789760775.002034:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705088478755 time 705106828446 state 0x0 ref 1 00000100:00100000:1.0:1789760775.002051:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705107088862 time 705106828446 state 0x1 ref 1 00000100:00100000:1.0:1789760775.002081:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705107088862 time 705126687403 state 0x0 ref 1 00000100:00100000:1.0:1789760775.002094:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.002145:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75751424, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24208 00010000:00010000:1.0:1789760775.002154:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.002170:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.002174:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.002181:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.002199:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.007974:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985801 is committed 00002000:00010000:1.0:1789760775.009196:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75751424 -> 75755520 00002000:00010000:1.0:1789760775.009204:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147976 -> 147984 00010000:00010000:1.0:1789760775.009218:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.009231:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.009241:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.009248:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.009252:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.009322:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.009501:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.009511:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75751424->75755519] (req 75751424->75755519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.009748:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503754880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7650us (8053us total) trans 4294985801 rc 0/0 00000100:00100000:1.0:1789760775.009760:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18505 00000100:00100000:1.0:1789760775.009770:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705107088862 time 705126687403 state 0x2 ref 0 00000100:00100000:0.0:1789760775.015845:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760775.027822:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985802 is committed 00000100:00100000:0.0:1789760775.032744:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503755136 00000100:00100000:0.0:1789760775.032754:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503755136 00000100:00100000:0.0:1789760775.032809:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705126947819 time 705142669502 state 0x0 ref 1 00000100:00100000:0.0:1789760775.032825:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705142929918 time 705142669502 state 0x1 ref 1 00000100:00100000:0.0:1789760775.032856:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705142929918 time 705157462680 state 0x0 ref 1 00000100:00100000:0.0:1789760775.032872:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503755136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.032920:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75759616, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24218 00010000:00010000:0.0:1789760775.032930:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.032946:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.032951:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.032958:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.032972:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.041997:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75759616 -> 75763712 00002000:00010000:1.0:1789760775.042001:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 147992 -> 148000 00010000:00010000:1.0:1789760775.042015:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.042026:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.042034:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.042041:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.042045:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.042053:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.042079:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.042093:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f01c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75759616->75763711] (req 75759616->75763711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.042183:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503755136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9309us (9875us total) trans 4294985803 rc 0/0 00000100:00100000:1.0:1789760775.042194:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18507 00000100:00100000:1.0:1789760775.042203:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705142929918 time 705157462680 state 0x2 ref 0 00000100:00100000:1.0:1789760775.050828:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.051180:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503755264 00000100:00100000:0.0:1789760775.051190:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503755264 00000100:00100000:0.0:1789760775.051249:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705142929918 time 705157462680 state 0x0 ref 1 00000100:00100000:0.0:1789760775.051264:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705157723096 time 705157462680 state 0x1 ref 1 00000100:00100000:0.0:1789760775.051296:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705157723096 time 705175902123 state 0x0 ref 1 00000100:00100000:0.0:1789760775.051309:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503755264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.051368:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75763712, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24220 00010000:00010000:0.0:1789760775.051378:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.051546:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.051551:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.051558:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.051572:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.058108:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75763712 -> 75767808 00002000:00010000:1.0:1789760775.058112:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148000 -> 148008 00010000:00010000:1.0:1789760775.058124:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.058136:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.058144:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.058151:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.058156:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.058164:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.058183:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.058190:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f01d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75763712->75767807] (req 75763712->75767807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.058283:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503755264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6972us (7460us total) trans 4294985804 rc 0/0 00000100:00100000:1.0:1789760775.058293:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18508 00000100:00100000:1.0:1789760775.058302:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705157723096 time 705175902123 state 0x2 ref 0 00000100:00100000:0.0:1789760775.061566:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760775.061667:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503755392 00000100:00100000:1.0:1789760775.061677:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503755392 00000100:00100000:1.0:1789760775.061733:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705157723096 time 705175902123 state 0x0 ref 1 00000100:00100000:1.0:1789760775.061748:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705176162539 time 705175902123 state 0x1 ref 1 00000100:00100000:1.0:1789760775.061781:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705176162539 time 705186387650 state 0x0 ref 1 00000100:00100000:1.0:1789760775.061794:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.061839:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75767808, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24228 00010000:00010000:1.0:1789760775.061848:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.061862:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.061868:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.061873:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.061886:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.073508:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75767808 -> 75771904 00002000:00010000:1.0:1789760775.073518:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148008 -> 148016 00010000:00010000:1.0:1789760775.073532:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.073544:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.073553:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.073562:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.073566:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.073575:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.073596:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.073610:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75767808->75771903] (req 75767808->75771903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.073840:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12041us (12278us total) trans 4294985805 rc 0/0 00000100:00100000:1.0:1789760775.073851:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18509 00000100:00100000:1.0:1789760775.073862:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705176162539 time 705186387650 state 0x2 ref 0 00000100:00100000:0.0:1789760775.083836:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503755520 00000100:00100000:0.0:1789760775.083847:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503755520 00000100:00100000:0.0:1789760775.083904:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705176162539 time 705186387650 state 0x0 ref 1 00000100:00100000:0.0:1789760775.083919:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705186648066 time 705186387650 state 0x1 ref 1 00000100:00100000:0.0:1789760775.083987:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705186648066 time 705208592444 state 0x0 ref 1 00000100:00100000:0.0:1789760775.083999:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.084046:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75771904, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24230 00010000:00010000:0.0:1789760775.084055:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.084079:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.084083:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.084090:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.084104:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.088763:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75771904 -> 75776000 00002000:00010000:0.0:1789760775.088769:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148016 -> 148024 00010000:00010000:0.0:1789760775.088782:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.088794:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.088803:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.088810:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.088815:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.088824:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.088843:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.088851:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75771904->75775999] (req 75771904->75775999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.089028:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5026us (5832us total) trans 4294985806 rc 0/0 00000100:00100000:0.0:1789760775.089040:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18510 00000100:00100000:0.0:1789760775.089050:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705186648066 time 705208592444 state 0x2 ref 0 00000100:00100000:1.0:1789760775.093579:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503755648 00000100:00100000:1.0:1789760775.093590:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503755648 00000100:00100000:1.0:1789760775.093656:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705186648066 time 705208592444 state 0x0 ref 1 00000100:00100000:1.0:1789760775.093671:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705208852860 time 705208592444 state 0x1 ref 1 00000100:00100000:1.0:1789760775.093701:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705208852860 time 705218308096 state 0x0 ref 1 00000100:00100000:1.0:1789760775.093712:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.093759:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75776000, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24238 00010000:00010000:1.0:1789760775.093768:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.093782:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.093787:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.093794:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.093811:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.100780:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75776000 -> 75780096 00002000:00010000:1.0:1789760775.100785:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148024 -> 148032 00010000:00010000:1.0:1789760775.100799:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.100812:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.100822:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.100829:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.100833:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.100842:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.100864:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.100871:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f01e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75776000->75780095] (req 75776000->75780095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.100971:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7256us (7964us total) trans 4294985807 rc 0/0 00000100:00100000:1.0:1789760775.100982:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18511 00000100:00100000:1.0:1789760775.100991:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705208852860 time 705218308096 state 0x2 ref 0 00000100:00100000:1.0:1789760775.106740:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503755776 00000100:00100000:1.0:1789760775.106750:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503755776 00000100:00100000:1.0:1789760775.106805:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705208852860 time 705218308096 state 0x0 ref 1 00000100:00100000:1.0:1789760775.106880:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705218568512 time 705218308096 state 0x1 ref 1 00000100:00100000:1.0:1789760775.106914:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705218568512 time 705231520288 state 0x0 ref 1 00000100:00100000:1.0:1789760775.106926:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503755776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.106986:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75780096, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24240 00010000:00010000:1.0:1789760775.106995:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.107020:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.107025:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.107031:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.107045:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.113467:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985808 is committed 00002000:00010000:1.0:1789760775.113648:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75780096 -> 75784192 00002000:00010000:1.0:1789760775.113709:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148032 -> 148040 00010000:00010000:1.0:1789760775.113725:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.113738:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.113747:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.113753:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.113757:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.113766:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.113785:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.113792:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75780096->75784191] (req 75780096->75784191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.113921:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503755776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6992us (7279us total) trans 4294985808 rc 0/0 00000100:00100000:1.0:1789760775.113932:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18512 00000100:00100000:1.0:1789760775.113942:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705218568512 time 705231520288 state 0x2 ref 0 00000100:00100000:1.0:1789760775.124198:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503755904 00000100:00100000:1.0:1789760775.124210:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503755904 00000100:00100000:1.0:1789760775.124310:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705218568512 time 705231520288 state 0x0 ref 1 00000100:00100000:1.0:1789760775.124325:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705231780704 time 705231520288 state 0x1 ref 1 00000100:00100000:1.0:1789760775.124483:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705231780704 time 705249088534 state 0x0 ref 1 00000100:00100000:1.0:1789760775.124496:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.124543:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75784192, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24248 00010000:00010000:1.0:1789760775.124551:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.124567:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.124572:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.124578:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.124591:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.129582:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985809 is committed 00002000:00010000:0.0:1789760775.132929:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75784192 -> 75788288 00002000:00010000:0.0:1789760775.132939:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148040 -> 148048 00010000:00010000:0.0:1789760775.132954:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.132968:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.132977:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.132985:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.132990:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.132999:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.133025:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.133032:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f01f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75784192->75788287] (req 75784192->75788287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.133129:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503755904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8630us (9288us total) trans 4294985809 rc 0/0 00000100:00100000:0.0:1789760775.133140:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18513 00000100:00100000:0.0:1789760775.133150:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705231780704 time 705249088534 state 0x2 ref 0 00000100:00100000:0.0:1789760775.138368:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503756032 00000100:00100000:0.0:1789760775.138382:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503756032 00000100:00100000:0.0:1789760775.138835:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705231780704 time 705249088534 state 0x0 ref 1 00000100:00100000:0.0:1789760775.138855:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705249348950 time 705249088534 state 0x1 ref 1 00000100:00100000:0.0:1789760775.138936:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705249348950 time 705263541112 state 0x0 ref 1 00000100:00100000:0.0:1789760775.138951:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503756032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.139024:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75788288, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24250 00010000:00010000:0.0:1789760775.139036:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.139059:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.139065:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.139072:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.139087:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.144656:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75788288 -> 75792384 00002000:00010000:0.0:1789760775.144661:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148048 -> 148056 00010000:00010000:0.0:1789760775.144674:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.144686:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.144695:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.144703:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.144707:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.144716:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.144736:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.144750:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f01fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75788288->75792383] (req 75788288->75792383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.144845:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503756032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5892us (6728us total) trans 4294985810 rc 0/0 00000100:00100000:0.0:1789760775.144856:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18514 00000100:00100000:0.0:1789760775.144865:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705249348950 time 705263541112 state 0x2 ref 0 00000100:00100000:1.0:1789760775.150501:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503756160 00000100:00100000:1.0:1789760775.150524:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503756160 00000100:00100000:1.0:1789760775.150590:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705249348950 time 705263541112 state 0x0 ref 1 00000100:00100000:1.0:1789760775.150605:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705263801528 time 705263541112 state 0x1 ref 1 00000100:00100000:1.0:1789760775.150781:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705263801528 time 705275385870 state 0x0 ref 1 00000100:00100000:1.0:1789760775.150796:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503756160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.150849:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75792384, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24258 00010000:00010000:1.0:1789760775.150858:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.150874:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.150879:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.150885:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.150899:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.155612:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75792384 -> 75796480 00002000:00010000:1.0:1789760775.155626:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148056 -> 148064 00010000:00010000:1.0:1789760775.155640:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.155653:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.155744:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.155752:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.155757:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.155766:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.155785:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.155792:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0201 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75792384->75796479] (req 75792384->75796479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.155887:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503756160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5088us (5557us total) trans 4294985811 rc 0/0 00000100:00100000:1.0:1789760775.155897:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18515 00000100:00100000:1.0:1789760775.155906:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705263801528 time 705275385870 state 0x2 ref 0 00000100:00100000:1.0:1789760775.158936:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503756288 00000100:00100000:1.0:1789760775.158945:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503756288 00000100:00100000:1.0:1789760775.159000:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705263801528 time 705275385870 state 0x0 ref 1 00000100:00100000:1.0:1789760775.159015:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705275646286 time 705275385870 state 0x1 ref 1 00000100:00100000:1.0:1789760775.159049:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705275646286 time 705283655462 state 0x0 ref 1 00000100:00100000:1.0:1789760775.159061:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503756288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.159225:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75796480, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24260 00010000:00010000:1.0:1789760775.159235:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.159251:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.159255:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.159261:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.159275:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.166557:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75796480 -> 75800576 00002000:00010000:0.0:1789760775.166562:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148064 -> 148072 00010000:00010000:0.0:1789760775.166574:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.166586:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.166596:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.166604:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.166608:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.166627:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.166653:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.166744:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0208 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75796480->75800575] (req 75796480->75800575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.166842:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503756288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7777us (8232us total) trans 4294985812 rc 0/0 00000100:00100000:0.0:1789760775.166853:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18516 00000100:00100000:0.0:1789760775.166863:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705275646286 time 705283655462 state 0x2 ref 0 00000100:00100000:0.0:1789760775.171101:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503756416 00000100:00100000:0.0:1789760775.171116:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503756416 00000100:00100000:0.0:1789760775.171177:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705275646286 time 705283655462 state 0x0 ref 1 00000100:00100000:0.0:1789760775.171193:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705283915878 time 705283655462 state 0x1 ref 1 00000100:00100000:0.0:1789760775.171503:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705283915878 time 705296106823 state 0x0 ref 1 00000100:00100000:0.0:1789760775.171518:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503756416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.171568:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75800576, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24268 00010000:00010000:0.0:1789760775.171577:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f020f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.171591:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.171596:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.171603:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f020f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75800576->75804671] (req 75800576->75804671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.171624:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f020f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75800576->75804671] (req 75800576->75804671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.176144:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985813 is committed 00000100:00100000:0.0:1789760775.179926:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503756544 00000100:00100000:0.0:1789760775.179937:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503756544 00000100:00100000:0.0:1789760775.179988:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705283915878 time 705296106823 state 0x0 ref 1 00000100:00100000:0.0:1789760775.180016:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705296367239 time 705296106823 state 0x1 ref 1 00000100:00100000:0.0:1789760775.180048:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705296367239 time 705304653926 state 0x0 ref 1 00000100:00100000:0.0:1789760775.180061:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503756544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.180105:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75804672, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24270 00010000:00010000:0.0:1789760775.180115:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180129:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.180134:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.180140:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180153:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.185559:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75804672 -> 75808768 00002000:00010000:0.0:1789760775.185564:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148080 -> 148088 00010000:00010000:0.0:1789760775.185577:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.185592:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.185601:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.185608:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.185612:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.185635:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.185655:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.185662:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0216 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75804672->75808767] (req 75804672->75808767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.185751:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503756544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5688us (5918us total) trans 4294985814 rc 0/0 00000100:00100000:0.0:1789760775.185762:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18518 00000100:00100000:0.0:1789760775.185772:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705296367239 time 705304653926 state 0x2 ref 0 00000100:00100000:1.0:1789760775.190282:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760775.195578:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985815 is committed 00000100:00100000:0.0:1789760775.201845:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503756800 00000100:00100000:0.0:1789760775.201855:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503756800 00000100:00100000:0.0:1789760775.201912:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705304914342 time 705315421376 state 0x0 ref 1 00000100:00100000:0.0:1789760775.201927:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705315681792 time 705315421376 state 0x1 ref 1 00000100:00100000:0.0:1789760775.201959:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705315681792 time 705326566189 state 0x0 ref 1 00000100:00100000:0.0:1789760775.201972:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503756800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.202035:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75812864, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24280 00010000:00010000:0.0:1789760775.202045:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.202060:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.202064:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.202070:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.202084:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.206757:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75812864 -> 75816960 00002000:00010000:0.0:1789760775.206763:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148096 -> 148104 00010000:00010000:0.0:1789760775.206777:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.206789:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.206798:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.206804:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.206809:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.206818:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.206836:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.206844:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0224 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75812864->75816959] (req 75812864->75816959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.206991:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503756800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5008us (5241us total) trans 4294985816 rc 0/0 00000100:00100000:0.0:1789760775.207002:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18520 00000100:00100000:0.0:1789760775.207020:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705315681792 time 705326566189 state 0x2 ref 0 00000100:00100000:0.0:1789760775.211214:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760775.217581:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985817 is committed 00000100:00100000:1.0:1789760775.221833:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760775.226742:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985818 is committed 00000100:00100000:1.0:1789760775.230022:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760775.237013:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985819 is committed 00000100:00100000:0.0:1789760775.246436:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760775.253043:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985820 is committed 00000100:00100000:1.0:1789760775.258474:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.258581:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503757440 00000100:00100000:0.0:1789760775.258592:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503757440 00000100:00100000:0.0:1789760775.258716:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705355104298 time 705372065508 state 0x0 ref 1 00000100:00100000:0.0:1789760775.258734:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705372325924 time 705372065508 state 0x1 ref 1 00000100:00100000:0.0:1789760775.258795:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705372325924 time 705383399862 state 0x0 ref 1 00000100:00100000:0.0:1789760775.258808:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503757440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.258854:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75833344, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x242a8 00010000:00010000:0.0:1789760775.258863:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.258879:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.258883:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.258890:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.258904:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.266640:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75833344 -> 75837440 00002000:00010000:0.0:1789760775.266645:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148136 -> 148144 00010000:00010000:0.0:1789760775.266667:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.266682:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.266690:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.266697:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.266702:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.266712:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.266732:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.266739:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0247 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75833344->75837439] (req 75833344->75837439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.266832:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503757440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8021us (8363us total) trans 4294985821 rc 0/0 00000100:00100000:0.0:1789760775.266843:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18525 00000100:00100000:0.0:1789760775.266853:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705372325924 time 705383399862 state 0x2 ref 0 00000100:00100000:1.0:1789760775.272207:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.272283:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503757568 00000100:00100000:0.0:1789760775.272293:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503757568 00000100:00100000:0.0:1789760775.272356:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705372325924 time 705383399862 state 0x0 ref 1 00000100:00100000:0.0:1789760775.272372:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705383660278 time 705383399862 state 0x1 ref 1 00000100:00100000:0.0:1789760775.272445:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705383660278 time 705397050768 state 0x0 ref 1 00000100:00100000:0.0:1789760775.272459:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503757568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.272506:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75837440, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x242b0 00010000:00010000:0.0:1789760775.272516:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.272531:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.272535:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.272541:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.272555:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.278429:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75837440 -> 75841536 00002000:00010000:0.0:1789760775.278434:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148144 -> 148152 00010000:00010000:0.0:1789760775.278448:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.278461:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.278470:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.278477:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.278482:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.278492:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.278510:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.278518:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f024e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75837440->75841535] (req 75837440->75841535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.278597:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503757568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6136us (6394us total) trans 4294985822 rc 0/0 00000100:00100000:0.0:1789760775.278607:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18526 00000100:00100000:0.0:1789760775.278628:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705383660278 time 705397050768 state 0x2 ref 0 00000100:00100000:1.0:1789760775.284316:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503757696 00000100:00100000:1.0:1789760775.284325:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503757696 00000100:00100000:1.0:1789760775.284523:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705383660278 time 705397050768 state 0x0 ref 1 00000100:00100000:1.0:1789760775.284540:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705397311184 time 705397050768 state 0x1 ref 1 00000100:00100000:1.0:1789760775.284604:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705397311184 time 705409209778 state 0x0 ref 1 00000100:00100000:1.0:1789760775.284626:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503757696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.284684:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75841536, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x242b8 00010000:00010000:1.0:1789760775.284694:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.284709:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.284713:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.284719:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.284733:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.289650:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75841536 -> 75845632 00002000:00010000:1.0:1789760775.289665:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148152 -> 148160 00010000:00010000:1.0:1789760775.289678:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.289691:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.289699:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.289713:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.289718:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.289727:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.289746:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.289754:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0255 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75841536->75845631] (req 75841536->75845631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.289886:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503757696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5266us (5664us total) trans 4294985823 rc 0/0 00000100:00100000:1.0:1789760775.289896:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18527 00000100:00100000:1.0:1789760775.289906:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705397311184 time 705409209778 state 0x2 ref 0 00000100:00100000:1.0:1789760775.294167:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:0.0:1789760775.301217:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985824 is committed 00000100:00100000:1.0:1789760775.304491:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.304581:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503757952 00000100:00100000:0.0:1789760775.304589:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503757952 00000100:00100000:0.0:1789760775.304652:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705409470194 time 705419247114 state 0x0 ref 1 00000100:00100000:0.0:1789760775.304667:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705419507530 time 705419247114 state 0x1 ref 1 00000100:00100000:0.0:1789760775.304697:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705419507530 time 705429303491 state 0x0 ref 1 00000100:00100000:0.0:1789760775.304709:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503757952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.304755:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75849728, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x242c8 00010000:00010000:0.0:1789760775.304764:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.304779:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.304784:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.304791:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.304803:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.309743:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75849728 -> 75853824 00002000:00010000:0.0:1789760775.309748:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148168 -> 148176 00010000:00010000:0.0:1789760775.309761:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309779:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309788:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309795:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.309800:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309809:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309828:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309836:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0263 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75849728->75853823] (req 75849728->75853823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.309977:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503757952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5264us (5489us total) trans 4294985825 rc 0/0 00000100:00100000:0.0:1789760775.309997:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18529 00000100:00100000:0.0:1789760775.310007:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705419507530 time 705429303491 state 0x2 ref 0 00000100:00100000:0.0:1789760775.314072:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00002000:00010000:0.0:1789760775.320510:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75853824 -> 75857920 00002000:00010000:0.0:1789760775.320514:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148176 -> 148184 00010000:00010000:0.0:1789760775.320525:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f026a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75853824->75857919] (req 75853824->75857919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.320536:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f026a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75853824->75857919] (req 75853824->75857919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.320542:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f026a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75853824->75857919] (req 75853824->75857919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.320547:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.320550:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f026a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75853824->75857919] (req 75853824->75857919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.320556:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f026a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75853824->75857919] (req 75853824->75857919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.320570:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f026a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75853824->75857919] (req 75853824->75857919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.320575:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f026a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75853824->75857919] (req 75853824->75857919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.320673:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503758080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6362us (6687us total) trans 4294985826 rc 0/0 00000100:00100000:0.0:1789760775.320683:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18530 00000100:00100000:0.0:1789760775.320694:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705429563907 time 705438902765 state 0x2 ref 0 00000100:00100000:1.0:1789760775.323907:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760775.328998:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985827 is committed 00000100:00100000:1.0:1789760775.334765:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.335025:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503758336 00000100:00100000:0.0:1789760775.335037:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503758336 00000100:00100000:0.0:1789760775.335102:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705439163181 time 705448823787 state 0x0 ref 1 00000100:00100000:0.0:1789760775.335116:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705449084203 time 705448823787 state 0x1 ref 1 00000100:00100000:0.0:1789760775.335150:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705449084203 time 705459756412 state 0x0 ref 1 00000100:00100000:0.0:1789760775.335163:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503758336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.335211:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75862016, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x242e0 00010000:00010000:0.0:1789760775.335221:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.335237:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.335241:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.335248:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.335262:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.341690:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985828 is committed 00002000:00010000:1.0:1789760775.342195:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75862016 -> 75866112 00002000:00010000:1.0:1789760775.342200:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148192 -> 148200 00010000:00010000:1.0:1789760775.342215:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.342228:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.342237:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.342245:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.342250:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.342258:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.342276:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.342283:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f0278 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75862016->75866111] (req 75862016->75866111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.342672:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503758336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7504us (7910us total) trans 4294985828 rc 0/0 00000100:00100000:1.0:1789760775.342682:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18532 00000100:00100000:1.0:1789760775.342693:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705449084203 time 705459756412 state 0x2 ref 0 00000100:00100000:0.0:1789760775.346056:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760775.355535:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985829 is committed 00000100:00100000:0.0:1789760775.361752:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503758592 00000100:00100000:0.0:1789760775.361763:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503758592 00000100:00100000:0.0:1789760775.361819:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705460016828 time 705471139428 state 0x0 ref 1 00000100:00100000:0.0:1789760775.361835:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705471399844 time 705471139428 state 0x1 ref 1 00000100:00100000:0.0:1789760775.361907:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705471399844 time 705486513158 state 0x0 ref 1 00000100:00100000:0.0:1789760775.361921:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc17450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503758592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.361968:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75870208, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x242f0 00010000:00010000:0.0:1789760775.361978:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.361994:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.361998:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.362004:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.362017:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.369474:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75870208 -> 75874304 00002000:00010000:0.0:1789760775.369481:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148208 -> 148216 00010000:00010000:0.0:1789760775.369496:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.369511:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.369520:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.369528:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.369533:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.369543:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.369562:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.369569:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0286 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75870208->75874303] (req 75870208->75874303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.369673:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc17450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503758592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7749us (8076us total) trans 4294985830 rc 0/0 00000100:00100000:0.0:1789760775.369683:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18534 00000100:00100000:0.0:1789760775.369693:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705471399844 time 705486513158 state 0x2 ref 0 00000100:00100000:1.0:1789760775.372959:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760775.379350:0:3636:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760775.379373:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc95f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523607808:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760775.379464:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760775.379525:0:8432:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607808 00000100:00100000:1.0:1789760775.379531:0:8432:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607808 00000100:00100000:1.0:1789760775.379577:0:8432:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789760775.379585:0:8432:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523607808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760775.379605:0:8432:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3847794688 avail 3615211520 00000100:00100000:1.0:1789760775.379653:0:8432:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523607808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (189us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760775.379660:0:8432:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789760775.379711:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc95f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523607808:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1789760775.380317:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760775.380370:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760775.380382:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cce83c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523607936:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760775.380445:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760775.380483:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523607936 00000100:00100000:3.0:1789760775.380488:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523607936 00000100:00100000:3.0:1789760775.380522:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1789760775.380529:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523607936:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760775.380544:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3847794688 avail 3615182848 00000100:00100000:3.0:1789760775.380578:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523607936:12345-0@lo:13:osp-pre-0-1.0 Request processed in 49us (132us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760775.380584:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00080000:3.0:1789760775.380600:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760775.380634:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cce83c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523607936:0@lo:13:osp-pre-0-1.0 00002000:00010000:1.0:1789760775.380765:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75874304 -> 75878400 00002000:00010000:1.0:1789760775.380769:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148216 -> 148224 00010000:00010000:1.0:1789760775.380781:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f028d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75874304->75878399] (req 75874304->75878399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.380793:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f028d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75874304->75878399] (req 75874304->75878399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.380802:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f028d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75874304->75878399] (req 75874304->75878399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.380809:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.380814:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f028d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75874304->75878399] (req 75874304->75878399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.380823:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f028d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75874304->75878399] (req 75874304->75878399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.380842:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f028d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75874304->75878399] (req 75874304->75878399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.380851:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f028d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75874304->75878399] (req 75874304->75878399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.380917:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503758720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7615us (7964us total) trans 4294985831 rc 0/0 00000100:00100000:1.0:1789760775.380926:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18535 00000100:00100000:1.0:1789760775.380934:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705486773574 time 705497896230 state 0x2 ref 0 00000100:00100000:1.0:1789760775.385671:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503758848 00000100:00100000:1.0:1789760775.385749:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503758848 00000100:00100000:1.0:1789760775.385878:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705486773574 time 705497896230 state 0x0 ref 1 00000100:00100000:1.0:1789760775.385895:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705498156646 time 705497896230 state 0x1 ref 1 00000100:00100000:1.0:1789760775.385928:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705498156646 time 705510534560 state 0x0 ref 1 00000100:00100000:1.0:1789760775.385941:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503758848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.385988:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75878400, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24300 00010000:00010000:1.0:1789760775.385997:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.386014:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.386018:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.386025:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.386038:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.390981:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75878400 -> 75882496 00002000:00010000:1.0:1789760775.390986:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148224 -> 148232 00010000:00010000:1.0:1789760775.390998:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.391010:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.391018:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.391025:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.391029:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.391038:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.391063:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.391071:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0294 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75878400->75882495] (req 75878400->75882495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.391170:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503758848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5225us (6207us total) trans 4294985832 rc 0/0 00000100:00100000:1.0:1789760775.391180:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18536 00000100:00100000:1.0:1789760775.391189:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705498156646 time 705510534560 state 0x2 ref 0 00000100:00100000:3.0:1789760775.394995:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.395190:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503758976 00000100:00100000:0.0:1789760775.395201:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503758976 00000100:00100000:0.0:1789760775.395257:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705498156646 time 705510534560 state 0x0 ref 1 00000100:00100000:0.0:1789760775.395272:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705510794976 time 705510534560 state 0x1 ref 1 00000100:00100000:0.0:1789760775.395302:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705510794976 time 705519908304 state 0x0 ref 1 00000100:00100000:0.0:1789760775.395316:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503758976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.395369:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75882496, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24308 00010000:00010000:0.0:1789760775.395378:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.395429:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.395433:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.395439:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.395452:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.401325:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75882496 -> 75886592 00002000:00010000:0.0:1789760775.401330:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148232 -> 148240 00010000:00010000:0.0:1789760775.401355:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.401368:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.401376:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.401382:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.401423:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.401434:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.401453:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.401460:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f029b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75882496->75886591] (req 75882496->75886591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.401546:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503758976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6226us (6555us total) trans 4294985833 rc 0/0 00000100:00100000:0.0:1789760775.401556:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18537 00000100:00100000:0.0:1789760775.401565:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705510794976 time 705519908304 state 0x2 ref 0 00000100:00100000:0.0:1789760775.405015:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503759104 00000100:00100000:0.0:1789760775.405025:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503759104 00000100:00100000:0.0:1789760775.405747:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705510794976 time 705519908304 state 0x0 ref 1 00000100:00100000:0.0:1789760775.405764:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705520168720 time 705519908304 state 0x1 ref 1 00000100:00100000:0.0:1789760775.405786:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705520168720 time 705530393380 state 0x0 ref 1 00000100:00100000:0.0:1789760775.405801:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503759104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.405860:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75886592, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24310 00010000:00010000:0.0:1789760775.405870:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.405886:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.405891:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.405896:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.405909:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.412177:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75886592 -> 75890688 00002000:00010000:0.0:1789760775.412184:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148240 -> 148248 00010000:00010000:0.0:1789760775.412196:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.412209:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.412218:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.412225:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.412230:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.412238:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.412260:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.412268:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75886592->75890687] (req 75886592->75890687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.412437:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503759104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6636us (7666us total) trans 4294985834 rc 0/0 00000100:00100000:0.0:1789760775.412449:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18538 00000100:00100000:0.0:1789760775.412459:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705520168720 time 705530393380 state 0x2 ref 0 00000100:00100000:3.0:1789760775.417217:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760775.417280:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503759232 00000100:00100000:1.0:1789760775.417291:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503759232 00000100:00100000:1.0:1789760775.417344:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705520168720 time 705530393380 state 0x0 ref 1 00000100:00100000:1.0:1789760775.417359:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705530653796 time 705530393380 state 0x1 ref 1 00000100:00100000:1.0:1789760775.417408:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705530653796 time 705541996347 state 0x0 ref 1 00000100:00100000:1.0:1789760775.417420:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503759232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.417466:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75890688, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24318 00010000:00010000:1.0:1789760775.417474:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.417489:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.417494:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.417500:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.417512:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.431510:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75890688 -> 75894784 00002000:00010000:1.0:1789760775.431522:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148248 -> 148256 00010000:00010000:1.0:1789760775.431538:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.431551:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.431559:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.431567:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.431572:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.431581:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.431601:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.431715:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75890688->75894783] (req 75890688->75894783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.431867:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503759232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14442us (14654us total) trans 4294985835 rc 0/0 00000100:00100000:1.0:1789760775.431878:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18539 00000100:00100000:1.0:1789760775.431888:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705530653796 time 705541996347 state 0x2 ref 0 00000100:00100000:1.0:1789760775.434611:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503759360 00000100:00100000:1.0:1789760775.434631:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503759360 00000100:00100000:1.0:1789760775.434682:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705530653796 time 705541996347 state 0x0 ref 1 00000100:00100000:1.0:1789760775.434697:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705542256763 time 705541996347 state 0x1 ref 1 00000100:00100000:1.0:1789760775.434718:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705542256763 time 705559324767 state 0x0 ref 1 00000100:00100000:1.0:1789760775.434730:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503759360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.434776:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75894784, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24320 00010000:00010000:1.0:1789760775.434791:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.434807:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.434811:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.434817:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.434835:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.445691:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75894784 -> 75898880 00002000:00010000:3.0:1789760775.445700:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148256 -> 148264 00010000:00010000:3.0:1789760775.445714:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.445726:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.445735:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.445743:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.445748:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.445757:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.445776:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.445784:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f02b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75894784->75898879] (req 75894784->75898879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.445890:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503759360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11157us (11334us total) trans 4294985836 rc 0/0 00000100:00100000:3.0:1789760775.445902:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18540 00000100:00100000:3.0:1789760775.445912:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705542256763 time 705559324767 state 0x2 ref 0 00000100:00100000:3.0:1789760775.459770:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.460037:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503759488 00000100:00100000:0.0:1789760775.460048:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503759488 00000100:00100000:0.0:1789760775.461455:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705542256763 time 705559324767 state 0x0 ref 1 00000100:00100000:0.0:1789760775.461472:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705559585183 time 705559324767 state 0x1 ref 1 00000100:00100000:0.0:1789760775.461513:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705559585183 time 705586119279 state 0x0 ref 1 00000100:00100000:0.0:1789760775.461529:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503759488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.461575:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75898880, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24328 00010000:00010000:0.0:1789760775.461584:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.461599:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.461605:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.461611:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.461645:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.469286:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985837 is committed 00002000:00010000:0.0:1789760775.469447:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75898880 -> 75902976 00002000:00010000:0.0:1789760775.469453:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148264 -> 148272 00010000:00010000:0.0:1789760775.469466:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.469479:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.469488:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.469495:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.469499:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.469509:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.469528:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.469541:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75898880->75902975] (req 75898880->75902975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.469638:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503759488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8109us (9872us total) trans 4294985837 rc 0/0 00000100:00100000:0.0:1789760775.469650:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18541 00000100:00100000:0.0:1789760775.469660:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705559585183 time 705586119279 state 0x2 ref 0 00000100:00100000:1.0:1789760775.472885:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.473014:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503759616 00000100:00100000:0.0:1789760775.473025:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503759616 00000100:00100000:0.0:1789760775.473077:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705559585183 time 705586119279 state 0x0 ref 1 00000100:00100000:0.0:1789760775.473092:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705586379695 time 705586119279 state 0x1 ref 1 00000100:00100000:0.0:1789760775.473125:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705586379695 time 705597731977 state 0x0 ref 1 00000100:00100000:0.0:1789760775.473137:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503759616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.473182:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75902976, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24330 00010000:00010000:0.0:1789760775.473191:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473207:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.473212:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.473218:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473236:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.477582:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985838 is committed 00002000:00010000:0.0:1789760775.477896:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75902976 -> 75907072 00002000:00010000:0.0:1789760775.477902:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148272 -> 148280 00010000:00010000:0.0:1789760775.477915:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477927:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477936:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477944:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.477948:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477957:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477976:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477983:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f02be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75902976->75907071] (req 75902976->75907071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.478208:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503759616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5067us (5327us total) trans 4294985838 rc 0/0 00000100:00100000:0.0:1789760775.478219:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18542 00000100:00100000:0.0:1789760775.478230:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705586379695 time 705597731977 state 0x2 ref 0 00000100:00100000:1.0:1789760775.489509:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503759744 00000100:00100000:1.0:1789760775.489523:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503759744 00000100:00100000:1.0:1789760775.489582:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705586379695 time 705597731977 state 0x0 ref 1 00000100:00100000:1.0:1789760775.489597:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705597992393 time 705597731977 state 0x1 ref 1 00000100:00100000:1.0:1789760775.489743:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705597992393 time 705614348311 state 0x0 ref 1 00000100:00100000:1.0:1789760775.489759:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503759744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.489806:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75907072, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24338 00010000:00010000:1.0:1789760775.489815:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.489830:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.489835:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.489841:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.489856:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.494749:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75907072 -> 75911168 00002000:00010000:1.0:1789760775.494755:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148280 -> 148288 00010000:00010000:1.0:1789760775.494768:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494780:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494789:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494796:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.494801:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494809:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494829:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494837:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f02c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75907072->75911167] (req 75907072->75911167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.494975:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503759744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5213us (7509us total) trans 4294985839 rc 0/0 00000100:00100000:1.0:1789760775.494987:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18543 00000100:00100000:1.0:1789760775.494997:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705597992393 time 705614348311 state 0x2 ref 0 00000100:00100000:3.0:1789760775.498549:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760775.505022:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985840 is committed 00000100:00100000:0.0:1789760775.509936:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503760000 00000100:00100000:0.0:1789760775.509947:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503760000 00000100:00100000:0.0:1789760775.510003:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705614608727 time 705623453604 state 0x0 ref 1 00000100:00100000:0.0:1789760775.510018:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705623714020 time 705623453604 state 0x1 ref 1 00000100:00100000:0.0:1789760775.510045:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705623714020 time 705634651706 state 0x0 ref 1 00000100:00100000:0.0:1789760775.510058:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503760000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.510106:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75915264, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24348 00010000:00010000:0.0:1789760775.510115:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.510130:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.510134:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.510140:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.510153:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.517155:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985841 is committed 00002000:00010000:0.0:1789760775.517494:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75915264 -> 75919360 00002000:00010000:0.0:1789760775.517499:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148296 -> 148304 00010000:00010000:0.0:1789760775.517513:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.517524:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.517532:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.517538:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.517543:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.517552:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.517571:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.517578:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f02d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75915264->75919359] (req 75915264->75919359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.518055:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503760000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7994us (8233us total) trans 4294985841 rc 0/0 00000100:00100000:0.0:1789760775.518066:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18545 00000100:00100000:0.0:1789760775.518076:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705623714020 time 705634651706 state 0x2 ref 0 00000100:00100000:0.0:1789760775.525511:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760775.531159:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985842 is committed 00000100:00100000:3.0:1789760775.537669:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760775.538189:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503760256 00000100:00100000:1.0:1789760775.538204:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503760256 00000100:00100000:1.0:1789760775.538264:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705634912122 time 705650486145 state 0x0 ref 1 00000100:00100000:1.0:1789760775.538283:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705650746561 time 705650486145 state 0x1 ref 1 00000100:00100000:1.0:1789760775.538315:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705650746561 time 705662921565 state 0x0 ref 1 00000100:00100000:1.0:1789760775.538326:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503760256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.538383:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75923456, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24358 00010000:00010000:1.0:1789760775.538575:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.538591:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.538595:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.538602:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.538625:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.544492:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985843 is committed 00002000:00010000:1.0:1789760775.544831:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75923456 -> 75927552 00002000:00010000:1.0:1789760775.544837:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148312 -> 148320 00010000:00010000:1.0:1789760775.544849:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.544861:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.544870:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.544877:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.544882:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.544891:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.544910:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.544919:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f02e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75923456->75927551] (req 75923456->75927551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.545278:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503760256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6947us (7613us total) trans 4294985843 rc 0/0 00000100:00100000:1.0:1789760775.545289:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18547 00000100:00100000:1.0:1789760775.545299:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705650746561 time 705662921565 state 0x2 ref 0 00000100:00100000:1.0:1789760775.549231:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.549368:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503760384 00000100:00100000:3.0:1789760775.549377:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503760384 00000100:00100000:3.0:1789760775.549466:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705650746561 time 705662921565 state 0x0 ref 1 00000100:00100000:3.0:1789760775.549482:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705663181981 time 705662921565 state 0x1 ref 1 00000100:00100000:3.0:1789760775.549511:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705663181981 time 705674116850 state 0x0 ref 1 00000100:00100000:3.0:1789760775.549523:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503760384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.549568:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75927552, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24360 00010000:00010000:3.0:1789760775.549578:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.549594:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760775.549598:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.549604:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.549629:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.554919:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985844 is committed 00002000:00010000:3.0:1789760775.555174:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75927552 -> 75931648 00002000:00010000:3.0:1789760775.555179:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148320 -> 148328 00010000:00010000:3.0:1789760775.555191:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.555204:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.555212:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.555219:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.555224:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.555233:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.555252:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.555261:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f02e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75927552->75931647] (req 75927552->75931647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.555366:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503760384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5840us (6139us total) trans 4294985844 rc 0/0 00000100:00100000:3.0:1789760775.555378:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18548 00000100:00100000:3.0:1789760775.555422:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705663181981 time 705674116850 state 0x2 ref 0 00000100:00100000:0.0:1789760775.563540:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760775.569356:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985845 is committed 00000100:00100000:0.0:1789760775.574472:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760775.579597:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985846 is committed 00000100:00100000:1.0:1789760775.582792:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760775.589441:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985847 is committed 00000100:00100000:0.0:1789760775.594689:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.594807:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503760896 00000100:00100000:3.0:1789760775.594816:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503760896 00000100:00100000:3.0:1789760775.594869:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705699569319 time 705707638992 state 0x0 ref 1 00000100:00100000:3.0:1789760775.594883:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705707899408 time 705707638992 state 0x1 ref 1 00000100:00100000:3.0:1789760775.594957:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705707899408 time 705719562920 state 0x0 ref 1 00000100:00100000:3.0:1789760775.594970:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503760896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.595012:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75943936, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24380 00010000:00010000:3.0:1789760775.595022:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.595037:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760775.595041:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.595048:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.595061:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.599873:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985848 is committed 00002000:00010000:3.0:1789760775.600148:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75943936 -> 75948032 00002000:00010000:3.0:1789760775.600153:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148352 -> 148360 00010000:00010000:3.0:1789760775.600165:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.600179:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.600187:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.600195:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.600199:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.600208:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.600296:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.600305:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0304 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75943936->75948031] (req 75943936->75948031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.600625:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503760896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5643us (5933us total) trans 4294985848 rc 0/0 00000100:00100000:3.0:1789760775.600637:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18552 00000100:00100000:3.0:1789760775.600657:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705707899408 time 705719562920 state 0x2 ref 0 00000100:00100000:0.0:1789760775.603979:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.604027:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503761024 00000100:00100000:3.0:1789760775.604036:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503761024 00000100:00100000:3.0:1789760775.604198:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705707899408 time 705719562920 state 0x0 ref 1 00000100:00100000:3.0:1789760775.604209:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705719823336 time 705719562920 state 0x1 ref 1 00000100:00100000:3.0:1789760775.604239:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705719823336 time 705728845651 state 0x0 ref 1 00000100:00100000:3.0:1789760775.604252:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503761024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.604362:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75948032, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24388 00010000:00010000:3.0:1789760775.604372:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.604387:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760775.604471:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.604477:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.604510:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.611056:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985849 is committed 00002000:00010000:0.0:1789760775.611260:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75948032 -> 75952128 00002000:00010000:0.0:1789760775.611265:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148360 -> 148368 00010000:00010000:0.0:1789760775.611278:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611289:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611298:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611304:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.611309:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611317:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611464:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611473:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f030b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75948032->75952127] (req 75948032->75952127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.611673:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503761024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7417us (7698us total) trans 4294985849 rc 0/0 00000100:00100000:0.0:1789760775.611683:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18553 00000100:00100000:0.0:1789760775.611693:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705719823336 time 705728845651 state 0x2 ref 0 00000100:00100000:1.0:1789760775.618114:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:1.0:1789760775.633104:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985850 is committed 00000100:00100000:1.0:1789760775.641795:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503761280 00000100:00100000:1.0:1789760775.641817:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503761280 00000100:00100000:1.0:1789760775.641876:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705729106067 time 705743130581 state 0x0 ref 1 00000100:00100000:1.0:1789760775.641892:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705743390997 time 705743130581 state 0x1 ref 1 00000100:00100000:1.0:1789760775.641925:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705743390997 time 705766531944 state 0x0 ref 1 00000100:00100000:1.0:1789760775.641938:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503761280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.642001:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75956224, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x24398 00010000:00010000:1.0:1789760775.642010:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0319 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.642027:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.642032:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.642039:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75956224->75960319] (req 75956224->75960319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.642059:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75956224->75960319] (req 75956224->75960319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.648449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985851 is committed 00000100:00100000:1.0:1789760775.658367:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.658503:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503761408 00000100:00100000:3.0:1789760775.658514:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503761408 00000100:00100000:3.0:1789760775.658577:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705743390997 time 705766531944 state 0x0 ref 1 00000100:00100000:3.0:1789760775.658592:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705766792360 time 705766531944 state 0x1 ref 1 00000100:00100000:3.0:1789760775.658627:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705766792360 time 705783233500 state 0x0 ref 1 00000100:00100000:3.0:1789760775.658640:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503761408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.658698:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75960320, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243a0 00010000:00010000:3.0:1789760775.658708:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.658723:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760775.658728:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.658735:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.658749:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.665380:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985852 is committed 00002000:00010000:3.0:1789760775.665809:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75960320 -> 75964416 00002000:00010000:3.0:1789760775.665815:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148384 -> 148392 00010000:00010000:3.0:1789760775.665829:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.665842:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.665856:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.665863:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.665867:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.665877:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.665896:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.666033:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0320 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75960320->75964415] (req 75960320->75964415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.666267:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503761408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7624us (7904us total) trans 4294985852 rc 0/0 00000100:00100000:3.0:1789760775.666277:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18556 00000100:00100000:3.0:1789760775.666287:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705766792360 time 705783233500 state 0x2 ref 0 00000100:00100000:1.0:1789760775.670999:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.671812:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503761536 00000100:00100000:0.0:1789760775.671821:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503761536 00000100:00100000:0.0:1789760775.671935:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705766792360 time 705783233500 state 0x0 ref 1 00000100:00100000:0.0:1789760775.671950:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705783493916 time 705783233500 state 0x1 ref 1 00000100:00100000:0.0:1789760775.671980:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705783493916 time 705796586966 state 0x0 ref 1 00000100:00100000:0.0:1789760775.671992:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503761536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.672038:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75964416, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243a8 00010000:00010000:0.0:1789760775.672479:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.672496:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.672501:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.672662:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.672680:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.680428:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985853 is committed 00002000:00010000:0.0:1789760775.680721:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75964416 -> 75968512 00002000:00010000:0.0:1789760775.680726:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148392 -> 148400 00010000:00010000:0.0:1789760775.680738:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.680752:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.680761:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.680767:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.680771:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.680780:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.680799:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.680811:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0327 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75964416->75968511] (req 75964416->75968511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.680935:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503761536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8939us (9940us total) trans 4294985853 rc 0/0 00000100:00100000:0.0:1789760775.680945:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18557 00000100:00100000:0.0:1789760775.680955:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705783493916 time 705796586966 state 0x2 ref 0 00000100:00100000:3.0:1789760775.689985:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000001:00080000:3.0:1789760775.699478:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985854 is committed 00010000:00010000:2.0:1789760775.699854:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f032e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75968512->75972607] (req 75968512->75972607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.700647:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503761664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10408us (10664us total) trans 4294985854 rc 0/0 00000100:00100000:2.0:1789760775.700660:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18558 00000100:00100000:2.0:1789760775.700670:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705796847382 time 705814828214 state 0x2 ref 0 00000100:00100000:1.0:1789760775.706894:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.707803:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503761792 00000100:00100000:2.0:1789760775.707814:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503761792 00000100:00100000:2.0:1789760775.707872:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705796847382 time 705814828214 state 0x0 ref 1 00000100:00100000:2.0:1789760775.707887:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705815088630 time 705814828214 state 0x1 ref 1 00000100:00100000:2.0:1789760775.707953:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705815088630 time 705832559359 state 0x0 ref 1 00000100:00100000:2.0:1789760775.707967:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503761792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.708013:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75972608, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243b8 00010000:00010000:2.0:1789760775.708022:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.708038:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760775.708042:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.708049:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.708063:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.712772:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985855 is committed 00002000:00010000:2.0:1789760775.712886:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75972608 -> 75976704 00002000:00010000:2.0:1789760775.712892:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148408 -> 148416 00010000:00010000:2.0:1789760775.712904:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.712917:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.712926:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.712933:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.712938:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.712946:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.712966:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.712974:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0335 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75972608->75976703] (req 75972608->75976703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.713061:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503761792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5093us (6174us total) trans 4294985855 rc 0/0 00000100:00100000:2.0:1789760775.713072:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18559 00000100:00100000:2.0:1789760775.713083:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705815088630 time 705832559359 state 0x2 ref 0 00000100:00100000:1.0:1789760775.717018:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.717175:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503761920 00000100:00100000:3.0:1789760775.717186:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503761920 00000100:00100000:3.0:1789760775.717253:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705815088630 time 705832559359 state 0x0 ref 1 00000100:00100000:3.0:1789760775.717273:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705832819775 time 705832559359 state 0x1 ref 1 00000100:00100000:3.0:1789760775.717305:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705832819775 time 705841909690 state 0x0 ref 1 00000100:00100000:3.0:1789760775.717317:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503761920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.717378:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75976704, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243c0 00010000:00010000:3.0:1789760775.717504:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717523:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760775.717600:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.717607:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717633:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.722847:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985856 is committed 00002000:00010000:2.0:1789760775.722956:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75976704 -> 75980800 00002000:00010000:2.0:1789760775.722961:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148416 -> 148424 00010000:00010000:2.0:1789760775.722974:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722987:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722995:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.723003:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.723008:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.723017:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.723036:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.723044:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f033c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75976704->75980799] (req 75976704->75980799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.723944:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503761920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6619us (6928us total) trans 4294985856 rc 0/0 00000100:00100000:2.0:1789760775.723957:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18560 00000100:00100000:2.0:1789760775.723967:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705832819775 time 705841909690 state 0x2 ref 0 00000100:00100000:3.0:1789760775.726514:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.728840:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762048 00000100:00100000:0.0:1789760775.728972:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762048 00000100:00100000:0.0:1789760775.729031:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705832819775 time 705841909690 state 0x0 ref 1 00000100:00100000:0.0:1789760775.729138:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705842170106 time 705841909690 state 0x1 ref 1 00000100:00100000:0.0:1789760775.729172:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705842170106 time 705853778065 state 0x0 ref 1 00000100:00100000:0.0:1789760775.729186:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.729231:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75980800, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243c8 00010000:00010000:0.0:1789760775.729240:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.729256:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.729260:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.729266:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.729283:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.739246:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985857 is committed 00002000:00010000:0.0:1789760775.739456:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75980800 -> 75984896 00002000:00010000:0.0:1789760775.739462:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148424 -> 148432 00010000:00010000:0.0:1789760775.739475:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.739489:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.739498:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.739506:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.739511:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.739520:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.739620:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.739630:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0343 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75980800->75984895] (req 75980800->75984895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.739877:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10688us (13369us total) trans 4294985857 rc 0/0 00000100:00100000:0.0:1789760775.739888:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18561 00000100:00100000:0.0:1789760775.739898:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705842170106 time 705853778065 state 0x2 ref 0 00000100:00100000:1.0:1789760775.744016:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.744298:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762176 00000100:00100000:2.0:1789760775.744310:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762176 00000100:00100000:2.0:1789760775.744371:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705842170106 time 705853778065 state 0x0 ref 1 00000100:00100000:2.0:1789760775.744387:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705854038481 time 705853778065 state 0x1 ref 1 00000100:00100000:2.0:1789760775.744731:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705854038481 time 705869335281 state 0x0 ref 1 00000100:00100000:2.0:1789760775.744745:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503762176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.744793:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75984896, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243d0 00010000:00010000:2.0:1789760775.744803:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.744819:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760775.744824:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.744830:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.744843:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.750363:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985858 is committed 00002000:00010000:2.0:1789760775.750576:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75984896 -> 75988992 00002000:00010000:2.0:1789760775.750580:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148432 -> 148440 00010000:00010000:2.0:1789760775.750593:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.750606:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.750631:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.750638:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.750643:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.750653:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.750672:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.750681:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f034a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75984896->75988991] (req 75984896->75988991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.750770:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503762176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6023us (6758us total) trans 4294985858 rc 0/0 00000100:00100000:2.0:1789760775.750781:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18562 00000100:00100000:2.0:1789760775.750791:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705854038481 time 705869335281 state 0x2 ref 0 00000100:00100000:1.0:1789760775.753600:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.753680:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762304 00000100:00100000:2.0:1789760775.753690:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762304 00000100:00100000:2.0:1789760775.753743:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705854038481 time 705869335281 state 0x0 ref 1 00000100:00100000:2.0:1789760775.753759:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705869595697 time 705869335281 state 0x1 ref 1 00000100:00100000:2.0:1789760775.753790:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705869595697 time 705878395991 state 0x0 ref 1 00000100:00100000:2.0:1789760775.753802:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.753851:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75988992, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243d8 00010000:00010000:2.0:1789760775.753859:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.753875:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760775.753879:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.753886:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.753899:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.757783:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985859 is committed 00002000:00010000:2.0:1789760775.758309:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75988992 -> 75993088 00002000:00010000:2.0:1789760775.758316:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148440 -> 148448 00010000:00010000:2.0:1789760775.758330:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.758910:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.758920:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.758929:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.758934:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.758944:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.758964:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.758973:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0351 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75988992->75993087] (req 75988992->75993087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.759069:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5264us (5473us total) trans 4294985859 rc 0/0 00000100:00100000:2.0:1789760775.759080:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18563 00000100:00100000:2.0:1789760775.759089:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705869595697 time 705878395991 state 0x2 ref 0 00000100:00100000:3.0:1789760775.762799:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.762863:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762432 00000100:00100000:2.0:1789760775.762873:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762432 00000100:00100000:2.0:1789760775.762926:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705869595697 time 705878395991 state 0x0 ref 1 00000100:00100000:2.0:1789760775.762941:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705878656407 time 705878395991 state 0x1 ref 1 00000100:00100000:2.0:1789760775.762975:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705878656407 time 705887581194 state 0x0 ref 1 00000100:00100000:2.0:1789760775.762989:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.763035:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75993088, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243e0 00010000:00010000:2.0:1789760775.763044:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.763069:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760775.763074:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.763081:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.763094:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.770047:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985860 is committed 00002000:00010000:2.0:1789760775.770154:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75993088 -> 75997184 00002000:00010000:2.0:1789760775.770159:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148448 -> 148456 00010000:00010000:2.0:1789760775.770172:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.770192:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.770201:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.770209:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.770214:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.770223:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.770242:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.770250:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0358 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75993088->75997183] (req 75993088->75997183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.770437:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7445us (7641us total) trans 4294985860 rc 0/0 00000100:00100000:2.0:1789760775.770448:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18564 00000100:00100000:2.0:1789760775.770457:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705878656407 time 705887581194 state 0x2 ref 0 00000100:00100000:1.0:1789760775.776040:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.777909:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762560 00000100:00100000:0.0:1789760775.777924:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762560 00000100:00100000:1.0:1789760775.779290:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705878656407 time 705887581194 state 0x0 ref 1 00000100:00100000:1.0:1789760775.779307:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705887841610 time 705887581194 state 0x1 ref 1 00000100:00100000:1.0:1789760775.779432:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705887841610 time 705904037544 state 0x0 ref 1 00000100:00100000:1.0:1789760775.779447:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503762560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.779500:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 75997184, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x243e8 00010000:00010000:1.0:1789760775.779521:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.779542:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760775.779546:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.779552:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.779566:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.784815:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985861 is committed 00002000:00010000:1.0:1789760775.785035:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 75997184 -> 76001280 00002000:00010000:1.0:1789760775.785039:0:8434:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:1.0:1789760775.785041:0:8434:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00002000:00010000:1.0:1789760775.785043:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148456 -> 148464 00010000:00010000:1.0:1789760775.785055:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.785066:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.785075:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.785082:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.785087:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.785097:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.785116:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.785121:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f035f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [75997184->76001279] (req 75997184->76001279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.785554:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503762560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5917us (9329us total) trans 4294985861 rc 0/0 00000100:00100000:1.0:1789760775.785574:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18565 00000100:00100000:1.0:1789760775.785595:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705887841610 time 705904037544 state 0x2 ref 0 00000100:00100000:1.0:1789760775.796387:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.796542:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762688 00000100:00100000:0.0:1789760775.796553:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762688 00000100:00100000:0.0:1789760775.796636:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705887841610 time 705904037544 state 0x0 ref 1 00000100:00100000:0.0:1789760775.796659:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705904297960 time 705904037544 state 0x1 ref 1 00000100:00100000:0.0:1789760775.796690:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705904297960 time 705921296326 state 0x0 ref 1 00000100:00100000:0.0:1789760775.796711:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503762688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.796763:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76001280, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x243f0 00010000:00010000:0.0:1789760775.796772:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.796789:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.796793:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.796799:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.796819:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.802896:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985862 is committed 00002000:00010000:0.0:1789760775.803449:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76001280 -> 76005376 00002000:00010000:0.0:1789760775.803456:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148464 -> 148472 00010000:00010000:0.0:1789760775.803469:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.803480:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.803488:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.803495:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.803499:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.803507:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.803526:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.803534:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0366 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76001280->76005375] (req 76001280->76005375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.806746:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503762688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10027us (10360us total) trans 4294985862 rc 0/0 00000100:00100000:0.0:1789760775.806761:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18566 00000100:00100000:0.0:1789760775.806770:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705904297960 time 705921296326 state 0x2 ref 0 00000100:00100000:3.0:1789760775.812376:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.812502:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762816 00000100:00100000:2.0:1789760775.812514:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762816 00000100:00100000:2.0:1789760775.812571:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705904297960 time 705921296326 state 0x0 ref 1 00000100:00100000:2.0:1789760775.812589:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705921556742 time 705921296326 state 0x1 ref 1 00000100:00100000:2.0:1789760775.812638:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705921556742 time 705937243530 state 0x0 ref 1 00000100:00100000:2.0:1789760775.812657:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.812722:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76005376, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x243f8 00010000:00010000:2.0:1789760775.812733:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.812751:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760775.812757:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.812763:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.812777:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.820493:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985863 is committed 00002000:00010000:2.0:1789760775.820636:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76005376 -> 76009472 00002000:00010000:2.0:1789760775.820642:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148472 -> 148480 00010000:00010000:2.0:1789760775.820656:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.820668:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.820682:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.820689:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.820694:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.820702:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.820721:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.820728:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f036d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76005376->76009471] (req 76005376->76009471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.820824:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503762816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8169us (8453us total) trans 4294985863 rc 0/0 00000100:00100000:2.0:1789760775.820835:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18567 00000100:00100000:2.0:1789760775.820844:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705921556742 time 705937243530 state 0x2 ref 0 00000100:00100000:1.0:1789760775.827542:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.827647:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503762944 00000100:00100000:2.0:1789760775.827658:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503762944 00000100:00100000:2.0:1789760775.827714:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705921556742 time 705937243530 state 0x0 ref 1 00000100:00100000:2.0:1789760775.827729:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705937503946 time 705937243530 state 0x1 ref 1 00000100:00100000:2.0:1789760775.827796:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705937503946 time 705952402149 state 0x0 ref 1 00000100:00100000:2.0:1789760775.827809:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503762944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.827858:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76009472, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24400 00010000:00010000:2.0:1789760775.827871:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.827886:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760775.827890:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.827896:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.827910:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.833835:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985864 is committed 00002000:00010000:2.0:1789760775.833955:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76009472 -> 76013568 00002000:00010000:2.0:1789760775.833961:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148480 -> 148488 00010000:00010000:2.0:1789760775.833975:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.833988:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.833998:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.834011:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.834015:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.834024:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.834046:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.834054:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0374 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76009472->76013567] (req 76009472->76013567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.834146:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503762944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6334us (6608us total) trans 4294985864 rc 0/0 00000100:00100000:2.0:1789760775.834157:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18568 00000100:00100000:2.0:1789760775.834168:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705937503946 time 705952402149 state 0x2 ref 0 00000100:00100000:1.0:1789760775.838305:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.838501:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763072 00000100:00100000:3.0:1789760775.838511:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763072 00000100:00100000:3.0:1789760775.838566:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705937503946 time 705952402149 state 0x0 ref 1 00000100:00100000:3.0:1789760775.838581:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705952662565 time 705952402149 state 0x1 ref 1 00000100:00100000:3.0:1789760775.838620:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705952662565 time 705963216703 state 0x0 ref 1 00000100:00100000:3.0:1789760775.838633:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.838688:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76013568, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24408 00010000:00010000:3.0:1789760775.838698:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.838714:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760775.838718:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.838724:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.838737:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.847427:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985865 is committed 00002000:00010000:3.0:1789760775.848952:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76013568 -> 76017664 00002000:00010000:3.0:1789760775.848961:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148488 -> 148496 00010000:00010000:3.0:1789760775.848975:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.848988:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.848995:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.849003:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.849007:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.849016:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.849034:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.849042:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f037b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76013568->76017663] (req 76013568->76017663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.849144:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10508us (10843us total) trans 4294985865 rc 0/0 00000100:00100000:3.0:1789760775.849154:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18569 00000100:00100000:3.0:1789760775.849164:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705952662565 time 705963216703 state 0x2 ref 0 00000100:00100000:3.0:1789760775.854592:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.854784:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763200 00000100:00100000:0.0:1789760775.854794:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763200 00000100:00100000:0.0:1789760775.854850:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705952662565 time 705963216703 state 0x0 ref 1 00000100:00100000:0.0:1789760775.854871:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705963477119 time 705963216703 state 0x1 ref 1 00000100:00100000:0.0:1789760775.854903:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705963477119 time 705979509377 state 0x0 ref 1 00000100:00100000:0.0:1789760775.854916:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503763200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.854962:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76017664, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24410 00010000:00010000:0.0:1789760775.854971:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.854986:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760775.854990:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.854996:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.855009:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.865094:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985866 is committed 00002000:00010000:0.0:1789760775.865290:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76017664 -> 76021760 00002000:00010000:0.0:1789760775.865296:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148496 -> 148504 00010000:00010000:0.0:1789760775.865309:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.865322:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.865331:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.865349:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.865354:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.865362:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.865382:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.865422:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0382 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76017664->76021759] (req 76017664->76021759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.865510:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503763200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10592us (10922us total) trans 4294985866 rc 0/0 00000100:00100000:0.0:1789760775.865521:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18570 00000100:00100000:0.0:1789760775.865532:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705963477119 time 705979509377 state 0x2 ref 0 00000100:00100000:1.0:1789760775.869326:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.869406:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763328 00000100:00100000:2.0:1789760775.869418:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763328 00000100:00100000:2.0:1789760775.869480:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705963477119 time 705979509377 state 0x0 ref 1 00000100:00100000:2.0:1789760775.869495:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705979769793 time 705979509377 state 0x1 ref 1 00000100:00100000:2.0:1789760775.869525:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705979769793 time 705994131596 state 0x0 ref 1 00000100:00100000:2.0:1789760775.869540:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.869587:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76021760, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24418 00010000:00010000:2.0:1789760775.869596:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.869630:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760775.869635:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.869642:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.869655:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.875865:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985867 is committed 00002000:00010000:2.0:1789760775.875982:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76021760 -> 76025856 00002000:00010000:2.0:1789760775.875988:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148504 -> 148512 00010000:00010000:2.0:1789760775.876000:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876014:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876023:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876032:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.876037:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876045:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876065:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876073:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0389 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76021760->76025855] (req 76021760->76025855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.876207:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6665us (6884us total) trans 4294985867 rc 0/0 00000100:00100000:2.0:1789760775.876218:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18571 00000100:00100000:2.0:1789760775.876227:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705979769793 time 705994131596 state 0x2 ref 0 00000100:00100000:1.0:1789760775.884014:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.885268:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763456 00000100:00100000:2.0:1789760775.885284:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763456 00000100:00100000:2.0:1789760775.885343:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705979769793 time 705994131596 state 0x0 ref 1 00000100:00100000:2.0:1789760775.885359:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705994392012 time 705994131596 state 0x1 ref 1 00000100:00100000:2.0:1789760775.885546:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705994392012 time 706010150892 state 0x0 ref 1 00000100:00100000:2.0:1789760775.885561:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.885621:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76025856, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24420 00010000:00010000:2.0:1789760775.885630:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.885646:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760775.885651:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.885667:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.885684:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.894866:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985868 is committed 00002000:00010000:2.0:1789760775.895131:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76025856 -> 76029952 00002000:00010000:2.0:1789760775.895137:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148512 -> 148520 00010000:00010000:2.0:1789760775.895151:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.895161:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.895167:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.895171:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.895174:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.895180:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.895197:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.895202:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0390 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76025856->76029951] (req 76025856->76029951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.895352:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9789us (11342us total) trans 4294985868 rc 0/0 00000100:00100000:2.0:1789760775.895363:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18572 00000100:00100000:2.0:1789760775.895372:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705994392012 time 706010150892 state 0x2 ref 0 00000100:00100000:2.0:1789760775.898867:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.898968:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763584 00000100:00100000:3.0:1789760775.898977:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763584 00000100:00100000:3.0:1789760775.899030:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 705994392012 time 706010150892 state 0x0 ref 1 00000100:00100000:3.0:1789760775.899045:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706010411308 time 706010150892 state 0x1 ref 1 00000100:00100000:3.0:1789760775.899095:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706010411308 time 706023700616 state 0x0 ref 1 00000100:00100000:3.0:1789760775.899108:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503763584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.899154:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76029952, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24428 00010000:00010000:3.0:1789760775.899164:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.899179:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760775.899184:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.899190:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.899203:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.907487:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985869 is committed 00002000:00010000:2.0:1789760775.913649:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76029952 -> 76034048 00002000:00010000:2.0:1789760775.913658:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148520 -> 148528 00010000:00010000:2.0:1789760775.913672:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.913684:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.913693:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.913701:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.913705:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.913715:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.913735:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.913742:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0397 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76029952->76034047] (req 76029952->76034047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.913842:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503763584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14732us (14982us total) trans 4294985869 rc 0/0 00000100:00100000:2.0:1789760775.913851:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18573 00000100:00100000:2.0:1789760775.913860:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706010411308 time 706023700616 state 0x2 ref 0 00000100:00100000:1.0:1789760775.919557:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.919637:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763712 00000100:00100000:3.0:1789760775.919719:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763712 00000100:00100000:3.0:1789760775.919772:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706010411308 time 706023700616 state 0x0 ref 1 00000100:00100000:3.0:1789760775.919787:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706023961032 time 706023700616 state 0x1 ref 1 00000100:00100000:3.0:1789760775.919819:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706023961032 time 706044426007 state 0x0 ref 1 00000100:00100000:3.0:1789760775.919832:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.919879:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76034048, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24430 00010000:00010000:3.0:1789760775.919889:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.919903:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760775.919908:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.919913:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.919926:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.925329:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985870 is committed 00002000:00010000:3.0:1789760775.925438:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76034048 -> 76038144 00002000:00010000:3.0:1789760775.925442:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148528 -> 148536 00010000:00010000:3.0:1789760775.925452:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.925461:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.925467:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.925475:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.925478:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.925484:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.925498:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.925502:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f039e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76034048->76038143] (req 76034048->76038143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.925639:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503763712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5805us (6087us total) trans 4294985870 rc 0/0 00000100:00100000:3.0:1789760775.925654:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18574 00000100:00100000:3.0:1789760775.925665:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706023961032 time 706044426007 state 0x2 ref 0 00000100:00100000:1.0:1789760775.931315:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760775.931416:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763840 00000100:00100000:2.0:1789760775.931426:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763840 00000100:00100000:2.0:1789760775.931480:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706023961032 time 706044426007 state 0x0 ref 1 00000100:00100000:2.0:1789760775.931495:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706044686423 time 706044426007 state 0x1 ref 1 00000100:00100000:2.0:1789760775.931563:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706044686423 time 706056168545 state 0x0 ref 1 00000100:00100000:2.0:1789760775.931576:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503763840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.931644:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76038144, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24438 00010000:00010000:2.0:1789760775.931662:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.931681:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760775.931685:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.931691:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.931704:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.938132:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985871 is committed 00002000:00010000:1.0:1789760775.938244:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76038144 -> 76042240 00002000:00010000:1.0:1789760775.938250:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148536 -> 148544 00010000:00010000:1.0:1789760775.938263:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.938282:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.938291:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.938298:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760775.938303:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.938312:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.938331:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.938568:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f03a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76038144->76042239] (req 76038144->76042239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.938679:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503763840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7100us (7368us total) trans 4294985871 rc 0/0 00000100:00100000:1.0:1789760775.938690:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18575 00000100:00100000:1.0:1789760775.938699:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706044686423 time 706056168545 state 0x2 ref 0 00000100:00100000:2.0:1789760775.941430:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.941517:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503763968 00000100:00100000:3.0:1789760775.941526:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503763968 00000100:00100000:3.0:1789760775.941581:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706044686423 time 706056168545 state 0x0 ref 1 00000100:00100000:3.0:1789760775.941605:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706056428961 time 706056168545 state 0x1 ref 1 00000100:00100000:3.0:1789760775.941636:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706056428961 time 706066243208 state 0x0 ref 1 00000100:00100000:3.0:1789760775.941651:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503763968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.941699:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76042240, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24440 00010000:00010000:3.0:1789760775.941708:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.941724:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760775.941729:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.941735:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.941749:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.946880:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985872 is committed 00002000:00010000:3.0:1789760775.950451:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76042240 -> 76046336 00002000:00010000:3.0:1789760775.950460:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148544 -> 148552 00010000:00010000:3.0:1789760775.950474:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.950487:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.950495:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.950503:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.950508:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.950517:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.950536:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.950544:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76042240->76046335] (req 76042240->76046335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.950756:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503763968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9101us (9363us total) trans 4294985872 rc 0/0 00000100:00100000:3.0:1789760775.950767:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18576 00000100:00100000:3.0:1789760775.950776:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706056428961 time 706066243208 state 0x2 ref 0 00000100:00100000:3.0:1789760775.953962:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760775.954174:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764096 00000100:00100000:1.0:1789760775.954183:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764096 00000100:00100000:1.0:1789760775.954298:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706056428961 time 706066243208 state 0x0 ref 1 00000100:00100000:1.0:1789760775.954312:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706066503624 time 706066243208 state 0x1 ref 1 00000100:00100000:2.0:1789760775.954519:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706066503624 time 706079122434 state 0x0 ref 1 00000100:00100000:2.0:1789760775.954535:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760775.954583:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76046336, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24448 00010000:00010000:2.0:1789760775.954592:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.954608:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760775.954622:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.954629:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.954643:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.959719:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985873 is committed 00002000:00010000:2.0:1789760775.964227:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76046336 -> 76050432 00002000:00010000:2.0:1789760775.964235:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148552 -> 148560 00010000:00010000:2.0:1789760775.964249:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.964262:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.964271:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.964278:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.964283:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.964291:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.964310:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.964317:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f03b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76046336->76050431] (req 76046336->76050431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.964572:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10036us (10616us total) trans 4294985873 rc 0/0 00000100:00100000:2.0:1789760775.964584:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18577 00000100:00100000:2.0:1789760775.964594:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706066503624 time 706079122434 state 0x2 ref 0 00000100:00100000:1.0:1789760775.968984:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.969112:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764224 00000100:00100000:0.0:1789760775.969122:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764224 00000100:00100000:0.0:1789760775.969175:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706066503624 time 706079122434 state 0x0 ref 1 00000100:00100000:0.0:1789760775.969189:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706079382850 time 706079122434 state 0x1 ref 1 00000100:00100000:0.0:1789760775.969219:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706079382850 time 706093825518 state 0x0 ref 1 00000100:00100000:0.0:1789760775.969231:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503764224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.969276:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76050432, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24450 00010000:00010000:0.0:1789760775.969284:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.969300:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760775.969305:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.969312:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.969326:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.973269:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985874 is committed 00002000:00010000:0.0:1789760775.974070:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76050432 -> 76054528 00002000:00010000:0.0:1789760775.974076:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148560 -> 148568 00010000:00010000:0.0:1789760775.974089:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.974105:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.974114:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.974122:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760775.974126:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.974136:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.974155:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.974163:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f03ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76050432->76054527] (req 76050432->76054527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.974296:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503764224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5062us (5316us total) trans 4294985874 rc 0/0 00000100:00100000:0.0:1789760775.974307:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18578 00000100:00100000:0.0:1789760775.974316:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706079382850 time 706093825518 state 0x2 ref 0 00000100:00100000:3.0:1789760775.977084:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760775.977151:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764352 00000100:00100000:1.0:1789760775.977161:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764352 00000100:00100000:1.0:1789760775.977221:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706079382850 time 706093825518 state 0x0 ref 1 00000100:00100000:1.0:1789760775.977236:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706094085934 time 706093825518 state 0x1 ref 1 00000100:00100000:1.0:1789760775.977268:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706094085934 time 706101874614 state 0x0 ref 1 00000100:00100000:1.0:1789760775.977280:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760775.977325:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76054528, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24458 00010000:00010000:1.0:1789760775.977343:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.977359:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760775.977364:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.977370:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.977384:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.981623:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985875 is committed 00002000:00010000:2.0:1789760775.981790:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76054528 -> 76058624 00002000:00010000:2.0:1789760775.981796:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148568 -> 148576 00010000:00010000:2.0:1789760775.981813:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981828:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981838:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981846:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760775.981852:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981861:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981883:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.981891:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f03c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76054528->76058623] (req 76054528->76058623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.982061:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4778us (4982us total) trans 4294985875 rc 0/0 00000100:00100000:2.0:1789760775.982072:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18579 00000100:00100000:2.0:1789760775.982082:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706094085934 time 706101874614 state 0x2 ref 0 00000100:00100000:2.0:1789760775.989275:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760775.989457:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764480 00000100:00100000:3.0:1789760775.989467:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764480 00000100:00100000:3.0:1789760775.989520:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706094085934 time 706101874614 state 0x0 ref 1 00000100:00100000:3.0:1789760775.989533:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706102135030 time 706101874614 state 0x1 ref 1 00000100:00100000:3.0:1789760775.989619:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706102135030 time 706114214533 state 0x0 ref 1 00000100:00100000:3.0:1789760775.989634:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503764480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760775.989692:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76058624, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24460 00010000:00010000:3.0:1789760775.989701:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.989716:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760775.989721:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.989727:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.989740:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.994017:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985876 is committed 00002000:00010000:3.0:1789760775.994311:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76058624 -> 76062720 00002000:00010000:3.0:1789760775.994316:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148576 -> 148584 00010000:00010000:3.0:1789760775.994329:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.994342:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.994350:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.994357:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760775.994361:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.994374:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.994429:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.994438:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76058624->76062719] (req 76058624->76062719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.994526:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503764480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4890us (5256us total) trans 4294985876 rc 0/0 00000100:00100000:3.0:1789760775.994536:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18580 00000100:00100000:3.0:1789760775.994546:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706102135030 time 706114214533 state 0x2 ref 0 00000100:00100000:2.0:1789760775.996983:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760775.997459:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764608 00000100:00100000:0.0:1789760775.997470:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764608 00000100:00100000:0.0:1789760775.997526:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706102135030 time 706114214533 state 0x0 ref 1 00000100:00100000:0.0:1789760775.997541:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706114474949 time 706114214533 state 0x1 ref 1 00000100:00100000:0.0:1789760775.997604:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706114474949 time 706122210040 state 0x0 ref 1 00000100:00100000:0.0:1789760775.997630:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760775.997686:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76062720, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24468 00010000:00010000:0.0:1789760775.997695:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.997712:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760775.997717:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.997723:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.997736:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.002298:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985877 is committed 00002000:00010000:0.0:1789760776.002496:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76062720 -> 76066816 00002000:00010000:0.0:1789760776.002502:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148584 -> 148592 00010000:00010000:0.0:1789760776.002515:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.002527:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.002542:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.002549:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.002553:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.002563:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.002582:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.002590:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f03cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76062720->76066815] (req 76062720->76066815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.002784:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5152us (5804us total) trans 4294985877 rc 0/0 00000100:00100000:0.0:1789760776.002795:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18581 00000100:00100000:0.0:1789760776.002805:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706114474949 time 706122210040 state 0x2 ref 0 00000100:00100000:0.0:1789760776.009528:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.010437:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764736 00000100:00100000:3.0:1789760776.010448:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764736 00000100:00100000:3.0:1789760776.010529:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706114474949 time 706122210040 state 0x0 ref 1 00000100:00100000:3.0:1789760776.010546:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706122470456 time 706122210040 state 0x1 ref 1 00000100:00100000:3.0:1789760776.010578:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706122470456 time 706135184778 state 0x0 ref 1 00000100:00100000:3.0:1789760776.010591:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503764736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.010647:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76066816, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24470 00010000:00010000:3.0:1789760776.010657:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760776.010665:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789760776.010675:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.010680:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.010686:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.010699:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.014788:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985878 is committed 00002000:00010000:0.0:1789760776.014918:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76066816 -> 76070912 00002000:00010000:0.0:1789760776.014923:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148592 -> 148600 00010000:00010000:0.0:1789760776.014938:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.014951:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.014960:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.014967:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.014972:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.014981:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.015000:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.015008:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f03d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76066816->76070911] (req 76066816->76070911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.015099:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503764736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4506us (5575us total) trans 4294985878 rc 0/0 00000100:00100000:0.0:1789760776.015109:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18582 00000100:00100000:0.0:1789760776.015118:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706122470456 time 706135184778 state 0x2 ref 0 00000100:00100000:2.0:1789760776.023013:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760776.023094:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764864 00000100:00100000:1.0:1789760776.023105:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764864 00000100:00100000:1.0:1789760776.023158:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706122470456 time 706135184778 state 0x0 ref 1 00000100:00100000:1.0:1789760776.023173:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706135445194 time 706135184778 state 0x1 ref 1 00000100:00100000:1.0:1789760776.023205:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706135445194 time 706147811600 state 0x0 ref 1 00000100:00100000:1.0:1789760776.023217:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.023262:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76070912, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24478 00010000:00010000:1.0:1789760776.023271:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.023288:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760776.023293:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.023298:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.023311:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.026913:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985879 is committed 00002000:00010000:1.0:1789760776.027015:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76070912 -> 76075008 00002000:00010000:1.0:1789760776.027020:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148600 -> 148608 00010000:00010000:1.0:1789760776.027033:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.027046:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.027056:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.027064:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.027068:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.027077:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.027097:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.027104:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f03dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76070912->76075007] (req 76070912->76075007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.027190:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3968us (4181us total) trans 4294985879 rc 0/0 00000100:00100000:1.0:1789760776.027199:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18583 00000100:00100000:1.0:1789760776.027210:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706135445194 time 706147811600 state 0x2 ref 0 00000100:00100000:2.0:1789760776.030164:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.030335:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503764992 00000100:00100000:3.0:1789760776.030346:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503764992 00000100:00100000:3.0:1789760776.030428:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706135445194 time 706147811600 state 0x0 ref 1 00000100:00100000:3.0:1789760776.030443:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706148072016 time 706147811600 state 0x1 ref 1 00000100:00100000:3.0:1789760776.030476:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706148072016 time 706155082415 state 0x0 ref 1 00000100:00100000:3.0:1789760776.030488:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.030539:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76075008, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24480 00010000:00010000:3.0:1789760776.030548:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030563:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.030567:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.030573:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030591:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.036406:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985880 is committed 00002000:00010000:3.0:1789760776.036508:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76075008 -> 76079104 00002000:00010000:3.0:1789760776.036513:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148608 -> 148616 00010000:00010000:3.0:1789760776.036526:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036538:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036546:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036554:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.036558:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036567:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036585:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036593:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f03e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76075008->76079103] (req 76075008->76079103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.036761:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503764992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6269us (6601us total) trans 4294985880 rc 0/0 00000100:00100000:3.0:1789760776.036772:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18584 00000100:00100000:3.0:1789760776.036781:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706148072016 time 706155082415 state 0x2 ref 0 00000100:00100000:0.0:1789760776.041121:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.041187:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503765120 00000100:00100000:2.0:1789760776.041198:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503765120 00000100:00100000:2.0:1789760776.041272:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706148072016 time 706155082415 state 0x0 ref 1 00000100:00100000:2.0:1789760776.041287:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706155342831 time 706155082415 state 0x1 ref 1 00000100:00100000:2.0:1789760776.041321:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706155342831 time 706165925951 state 0x0 ref 1 00000100:00100000:2.0:1789760776.041333:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503765120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.041381:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76079104, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24488 00010000:00010000:2.0:1789760776.041423:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.041440:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.041445:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.041452:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.041564:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.046927:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985881 is committed 00002000:00010000:2.0:1789760776.047030:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76079104 -> 76083200 00002000:00010000:2.0:1789760776.047036:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148616 -> 148624 00010000:00010000:2.0:1789760776.047048:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.047061:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.047069:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.047076:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.047080:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.047089:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.047109:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.047123:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76079104->76083199] (req 76079104->76083199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.047264:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503765120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5928us (6148us total) trans 4294985881 rc 0/0 00000100:00100000:2.0:1789760776.047275:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18585 00000100:00100000:2.0:1789760776.047284:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706155342831 time 706165925951 state 0x2 ref 0 00000100:00100000:2.0:1789760776.050880:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.050966:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503765248 00000100:00100000:0.0:1789760776.050976:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503765248 00000100:00100000:0.0:1789760776.051027:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706155342831 time 706165925951 state 0x0 ref 1 00000100:00100000:0.0:1789760776.051131:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706166186367 time 706165925951 state 0x1 ref 1 00000100:00100000:0.0:1789760776.051193:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706166186367 time 706175799384 state 0x0 ref 1 00000100:00100000:0.0:1789760776.051206:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503765248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.051250:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76083200, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24490 00010000:00010000:0.0:1789760776.051259:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.051274:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.051279:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.051285:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.051298:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.055137:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985882 is committed 00002000:00010000:2.0:1789760776.055296:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76083200 -> 76087296 00002000:00010000:2.0:1789760776.055302:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148624 -> 148632 00010000:00010000:2.0:1789760776.055316:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.055328:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.055346:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.055355:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.055360:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.055369:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.055494:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.055504:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f03f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76083200->76087295] (req 76083200->76087295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.055758:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503765248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4549us (4882us total) trans 4294985882 rc 0/0 00000100:00100000:2.0:1789760776.055769:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18586 00000100:00100000:2.0:1789760776.055779:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706166186367 time 706175799384 state 0x2 ref 0 00000100:00100000:1.0:1789760776.058512:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.058601:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503765376 00000100:00100000:2.0:1789760776.058620:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503765376 00000100:00100000:2.0:1789760776.058684:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706166186367 time 706175799384 state 0x0 ref 1 00000100:00100000:2.0:1789760776.058698:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706176059800 time 706175799384 state 0x1 ref 1 00000100:00100000:2.0:1789760776.058727:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706176059800 time 706183333810 state 0x0 ref 1 00000100:00100000:2.0:1789760776.058740:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503765376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.058790:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76087296, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24498 00010000:00010000:2.0:1789760776.058800:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.058817:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.058822:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.058828:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.058842:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.062347:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985883 is committed 00002000:00010000:2.0:1789760776.062488:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76087296 -> 76091392 00002000:00010000:2.0:1789760776.062493:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148632 -> 148640 00010000:00010000:2.0:1789760776.062506:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.062517:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.062526:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.062602:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.062607:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.062630:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.062717:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.062732:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f03f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76087296->76091391] (req 76087296->76091391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.062821:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503765376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4080us (4314us total) trans 4294985883 rc 0/0 00000100:00100000:2.0:1789760776.062831:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18587 00000100:00100000:2.0:1789760776.062840:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706176059800 time 706183333810 state 0x2 ref 0 00000100:00100000:0.0:1789760776.065758:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.065846:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503765504 00000100:00100000:3.0:1789760776.065857:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503765504 00000100:00100000:3.0:1789760776.065909:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706176059800 time 706183333810 state 0x0 ref 1 00000100:00100000:3.0:1789760776.065924:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706183594226 time 706183333810 state 0x1 ref 1 00000100:00100000:3.0:1789760776.065956:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706183594226 time 706190562318 state 0x0 ref 1 00000100:00100000:3.0:1789760776.065969:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503765504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.066016:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76091392, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244a0 00010000:00010000:3.0:1789760776.066026:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.066041:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.066045:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.066051:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.066079:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.070291:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985884 is committed 00002000:00010000:3.0:1789760776.070438:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76091392 -> 76095488 00002000:00010000:3.0:1789760776.070443:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148640 -> 148648 00010000:00010000:3.0:1789760776.070453:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.070462:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.070468:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.070472:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.070475:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.070482:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.070496:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.070500:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0400 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76091392->76095487] (req 76091392->76095487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.070938:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503765504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4677us (4896us total) trans 4294985884 rc 0/0 00000100:00100000:3.0:1789760776.070950:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18588 00000100:00100000:3.0:1789760776.070960:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706183594226 time 706190562318 state 0x2 ref 0 00000100:00100000:3.0:1789760776.074840:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.074975:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503765632 00000100:00100000:0.0:1789760776.074984:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503765632 00000100:00100000:0.0:1789760776.075039:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706183594226 time 706190562318 state 0x0 ref 1 00000100:00100000:0.0:1789760776.075121:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706190822734 time 706190562318 state 0x1 ref 1 00000100:00100000:0.0:1789760776.075155:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706190822734 time 706199761202 state 0x0 ref 1 00000100:00100000:0.0:1789760776.075169:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503765632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.075215:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76095488, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244a8 00010000:00010000:0.0:1789760776.075224:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.075239:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.075243:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.075249:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.075263:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.079597:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985885 is committed 00002000:00010000:0.0:1789760776.079753:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76095488 -> 76099584 00002000:00010000:0.0:1789760776.079758:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148648 -> 148656 00010000:00010000:0.0:1789760776.079770:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.079782:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.079792:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.079860:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.079866:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.079876:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.079895:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.079903:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0407 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76095488->76099583] (req 76095488->76099583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.079987:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503765632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4816us (5154us total) trans 4294985885 rc 0/0 00000100:00100000:0.0:1789760776.079996:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18589 00000100:00100000:0.0:1789760776.080005:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706190822734 time 706199761202 state 0x2 ref 0 00000100:00100000:1.0:1789760776.083273:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.083372:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503765760 00000100:00100000:2.0:1789760776.083382:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503765760 00000100:00100000:2.0:1789760776.083635:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706190822734 time 706199761202 state 0x0 ref 1 00000100:00100000:2.0:1789760776.083706:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706200021618 time 706199761202 state 0x1 ref 1 00000100:00100000:3.0:1789760776.083764:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706200021618 time 706208368583 state 0x0 ref 1 00000100:00100000:3.0:1789760776.083778:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503765760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.083885:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76099584, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244b0 00010000:00010000:3.0:1789760776.083893:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.083909:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.083913:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.083919:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.083932:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.088716:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985886 is committed 00002000:00010000:2.0:1789760776.088938:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76099584 -> 76103680 00002000:00010000:2.0:1789760776.088944:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148656 -> 148664 00010000:00010000:2.0:1789760776.088957:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.088970:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.088980:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.088987:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.088992:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.089001:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.089021:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.089028:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f040e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76099584->76103679] (req 76099584->76103679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.089130:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503765760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5350us (5862us total) trans 4294985886 rc 0/0 00000100:00100000:2.0:1789760776.089140:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18590 00000100:00100000:2.0:1789760776.089150:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706200021618 time 706208368583 state 0x2 ref 0 00000100:00100000:3.0:1789760776.091678:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.091769:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503765888 00000100:00100000:2.0:1789760776.091780:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503765888 00000100:00100000:2.0:1789760776.091845:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706200021618 time 706208368583 state 0x0 ref 1 00000100:00100000:2.0:1789760776.091860:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706208628999 time 706208368583 state 0x1 ref 1 00000100:00100000:2.0:1789760776.091937:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706208628999 time 706216534657 state 0x0 ref 1 00000100:00100000:2.0:1789760776.091950:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc16450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503765888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.092005:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76103680, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244b8 00010000:00010000:2.0:1789760776.092014:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.092028:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.092033:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.092039:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.092053:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.095279:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985887 is committed 00002000:00010000:2.0:1789760776.095479:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76103680 -> 76107776 00002000:00010000:2.0:1789760776.095483:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148664 -> 148672 00010000:00010000:2.0:1789760776.095494:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.095503:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.095509:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.095516:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.095520:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.095529:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.095549:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.095557:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0415 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76103680->76107775] (req 76103680->76107775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.095682:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc16450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503765888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3729us (4007us total) trans 4294985887 rc 0/0 00000100:00100000:2.0:1789760776.095693:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18591 00000100:00100000:2.0:1789760776.095703:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706208628999 time 706216534657 state 0x2 ref 0 00000100:00100000:2.0:1789760776.098180:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.098349:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766016 00000100:00100000:0.0:1789760776.098359:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766016 00000100:00100000:0.0:1789760776.098494:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706208628999 time 706216534657 state 0x0 ref 1 00000100:00100000:0.0:1789760776.098509:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706216795073 time 706216534657 state 0x1 ref 1 00000100:00100000:0.0:1789760776.098539:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706216795073 time 706223145560 state 0x0 ref 1 00000100:00100000:0.0:1789760776.098552:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503766016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.098594:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76107776, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244c0 00010000:00010000:0.0:1789760776.098603:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.098628:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.098633:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.098640:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.098757:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.102973:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985888 is committed 00002000:00010000:0.0:1789760776.103106:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76107776 -> 76111872 00002000:00010000:0.0:1789760776.103112:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148672 -> 148680 00010000:00010000:0.0:1789760776.103123:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.103136:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.103142:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.103151:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.103155:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.103161:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.103177:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.103183:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f041c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76107776->76111871] (req 76107776->76111871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.103320:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503766016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4766us (5157us total) trans 4294985888 rc 0/0 00000100:00100000:0.0:1789760776.103330:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18592 00000100:00100000:0.0:1789760776.103348:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706216795073 time 706223145560 state 0x2 ref 0 00000100:00100000:1.0:1789760776.105690:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.105736:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766144 00000100:00100000:0.0:1789760776.105745:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766144 00000100:00100000:0.0:1789760776.105798:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706216795073 time 706223145560 state 0x0 ref 1 00000100:00100000:0.0:1789760776.105810:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706223405976 time 706223145560 state 0x1 ref 1 00000100:00100000:0.0:1789760776.105834:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706223405976 time 706230440625 state 0x0 ref 1 00000100:00100000:0.0:1789760776.105849:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.105891:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76111872, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244c8 00010000:00010000:0.0:1789760776.105901:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.105921:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.105925:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.105940:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.105955:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.110674:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985889 is committed 00002000:00010000:0.0:1789760776.110885:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76111872 -> 76115968 00002000:00010000:0.0:1789760776.110890:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148680 -> 148688 00010000:00010000:0.0:1789760776.110903:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.110915:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.110924:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.110931:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.110936:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.110945:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.110965:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.110972:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0423 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76111872->76115967] (req 76111872->76115967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.111062:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5210us (5376us total) trans 4294985889 rc 0/0 00000100:00100000:0.0:1789760776.111072:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18593 00000100:00100000:0.0:1789760776.111081:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706223405976 time 706230440625 state 0x2 ref 0 00000100:00100000:3.0:1789760776.115701:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.115970:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766272 00000100:00100000:2.0:1789760776.115980:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766272 00000100:00100000:2.0:1789760776.116032:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706223405976 time 706230440625 state 0x0 ref 1 00000100:00100000:2.0:1789760776.116047:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706230701041 time 706230440625 state 0x1 ref 1 00000100:00100000:2.0:1789760776.116107:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706230701041 time 706240712874 state 0x0 ref 1 00000100:00100000:2.0:1789760776.116121:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503766272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.116165:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76115968, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244d0 00010000:00010000:2.0:1789760776.116186:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.116206:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.116210:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.116217:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.116230:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.121951:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985890 is committed 00002000:00010000:2.0:1789760776.122107:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76115968 -> 76120064 00002000:00010000:2.0:1789760776.122112:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148688 -> 148696 00010000:00010000:2.0:1789760776.122124:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.122137:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.122145:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.122152:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.122156:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.122165:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.122255:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.122263:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f042a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76115968->76120063] (req 76115968->76120063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.122887:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503766272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6763us (7189us total) trans 4294985890 rc 0/0 00000100:00100000:2.0:1789760776.122897:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18594 00000100:00100000:2.0:1789760776.122907:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706230701041 time 706240712874 state 0x2 ref 0 00000100:00100000:2.0:1789760776.125177:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760776.125256:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766400 00000100:00100000:1.0:1789760776.125265:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766400 00000100:00100000:1.0:1789760776.125316:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706230701041 time 706240712874 state 0x0 ref 1 00000100:00100000:1.0:1789760776.125331:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706240973290 time 706240712874 state 0x1 ref 1 00000100:00100000:1.0:1789760776.125367:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706240973290 time 706249973163 state 0x0 ref 1 00000100:00100000:1.0:1789760776.125379:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.125468:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76120064, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244d8 00010000:00010000:1.0:1789760776.125478:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.125493:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760776.125497:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.125503:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.125524:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.129441:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985891 is committed 00002000:00010000:1.0:1789760776.129551:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76120064 -> 76124160 00002000:00010000:1.0:1789760776.129556:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148696 -> 148704 00010000:00010000:1.0:1789760776.129567:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.129580:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.129588:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.129595:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.129600:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.129621:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.129640:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.129711:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0431 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76120064->76124159] (req 76120064->76124159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.129994:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4612us (4828us total) trans 4294985891 rc 0/0 00000100:00100000:1.0:1789760776.130005:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18595 00000100:00100000:1.0:1789760776.130014:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706240973290 time 706249973163 state 0x2 ref 0 00000100:00100000:1.0:1789760776.133702:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.133895:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766528 00000100:00100000:0.0:1789760776.133904:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766528 00000100:00100000:0.0:1789760776.133958:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706240973290 time 706249973163 state 0x0 ref 1 00000100:00100000:0.0:1789760776.133989:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706250233579 time 706249973163 state 0x1 ref 1 00000100:00100000:0.0:1789760776.134021:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706250233579 time 706258627564 state 0x0 ref 1 00000100:00100000:0.0:1789760776.134033:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503766528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.134079:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76124160, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244e0 00010000:00010000:0.0:1789760776.134088:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.134102:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.134107:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.134113:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.134126:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.139044:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985892 is committed 00002000:00010000:2.0:1789760776.139279:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76124160 -> 76128256 00002000:00010000:2.0:1789760776.139286:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148704 -> 148712 00010000:00010000:2.0:1789760776.139300:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.139314:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.139324:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.139332:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.139337:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.139347:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.139370:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.139380:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0438 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76124160->76128255] (req 76124160->76128255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.139991:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503766528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5955us (6293us total) trans 4294985892 rc 0/0 00000100:00100000:2.0:1789760776.140003:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18596 00000100:00100000:2.0:1789760776.140012:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706250233579 time 706258627564 state 0x2 ref 0 00000100:00100000:3.0:1789760776.143230:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.143953:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766656 00000100:00100000:0.0:1789760776.143964:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766656 00000100:00100000:0.0:1789760776.144025:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706250233579 time 706258627564 state 0x0 ref 1 00000100:00100000:0.0:1789760776.144041:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706258887980 time 706258627564 state 0x1 ref 1 00000100:00100000:0.0:1789760776.144134:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706258887980 time 706268739299 state 0x0 ref 1 00000100:00100000:0.0:1789760776.144150:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.144197:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76128256, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244e8 00010000:00010000:0.0:1789760776.144207:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.144222:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.144227:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.144234:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.144250:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.150509:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985893 is committed 00002000:00010000:0.0:1789760776.150813:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76128256 -> 76132352 00002000:00010000:0.0:1789760776.150819:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148712 -> 148720 00010000:00010000:0.0:1789760776.150832:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.150845:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.150854:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.150861:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.150867:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.150875:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.150893:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.150900:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f043f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76128256->76132351] (req 76128256->76132351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.151025:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6874us (7799us total) trans 4294985893 rc 0/0 00000100:00100000:0.0:1789760776.151035:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18597 00000100:00100000:0.0:1789760776.151045:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706258887980 time 706268739299 state 0x2 ref 0 00000100:00100000:0.0:1789760776.156789:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.156963:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766784 00000100:00100000:2.0:1789760776.156973:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766784 00000100:00100000:2.0:1789760776.157025:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706258887980 time 706268739299 state 0x0 ref 1 00000100:00100000:2.0:1789760776.157039:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706268999715 time 706268739299 state 0x1 ref 1 00000100:00100000:2.0:1789760776.157288:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706268999715 time 706281893318 state 0x0 ref 1 00000100:00100000:2.0:1789760776.157302:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503766784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.157346:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76132352, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244f0 00010000:00010000:2.0:1789760776.157355:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.157369:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.157374:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.157380:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.157428:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.162641:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985894 is committed 00002000:00010000:2.0:1789760776.162819:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76132352 -> 76136448 00002000:00010000:2.0:1789760776.162824:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148720 -> 148728 00010000:00010000:2.0:1789760776.162838:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.162850:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.162858:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.162865:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.162869:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.162879:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.162909:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.162916:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0446 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76132352->76136447] (req 76132352->76136447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.163013:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503766784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5709us (6228us total) trans 4294985894 rc 0/0 00000100:00100000:2.0:1789760776.163022:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18598 00000100:00100000:2.0:1789760776.163032:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706268999715 time 706281893318 state 0x2 ref 0 00000100:00100000:1.0:1789760776.165833:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.165917:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503766912 00000100:00100000:2.0:1789760776.165928:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503766912 00000100:00100000:2.0:1789760776.165980:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706268999715 time 706281893318 state 0x0 ref 1 00000100:00100000:2.0:1789760776.165996:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706282153734 time 706281893318 state 0x1 ref 1 00000100:00100000:2.0:1789760776.166146:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706282153734 time 706290751047 state 0x0 ref 1 00000100:00100000:2.0:1789760776.166160:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.166205:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76136448, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x244f8 00010000:00010000:2.0:1789760776.166214:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.166238:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.166243:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.166250:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.166263:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.173769:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985895 is committed 00002000:00010000:2.0:1789760776.173944:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76136448 -> 76140544 00002000:00010000:2.0:1789760776.173949:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148728 -> 148736 00010000:00010000:2.0:1789760776.173960:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.173972:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.173981:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.173987:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.173992:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.174000:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.174019:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.174032:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f044d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76136448->76140543] (req 76136448->76140543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.174517:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503766912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8353us (8694us total) trans 4294985895 rc 0/0 00000100:00100000:2.0:1789760776.174527:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18599 00000100:00100000:2.0:1789760776.174537:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706282153734 time 706290751047 state 0x2 ref 0 00000100:00100000:3.0:1789760776.181574:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.181740:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767040 00000100:00100000:0.0:1789760776.181750:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767040 00000100:00100000:0.0:1789760776.181804:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706282153734 time 706290751047 state 0x0 ref 1 00000100:00100000:0.0:1789760776.181818:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706291011463 time 706290751047 state 0x1 ref 1 00000100:00100000:0.0:1789760776.181879:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706291011463 time 706306485614 state 0x0 ref 1 00000100:00100000:0.0:1789760776.181892:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503767040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.181935:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76140544, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24500 00010000:00010000:0.0:1789760776.181944:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.181957:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.181961:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.181967:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.181981:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.186947:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985896 is committed 00002000:00010000:0.0:1789760776.187056:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76140544 -> 76144640 00002000:00010000:0.0:1789760776.187061:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148736 -> 148744 00010000:00010000:0.0:1789760776.187085:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.187098:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.187107:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.187114:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.187119:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.187128:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.187148:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.187156:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0454 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76140544->76144639] (req 76140544->76144639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.187246:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503767040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5350us (5675us total) trans 4294985896 rc 0/0 00000100:00100000:0.0:1789760776.187257:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18600 00000100:00100000:0.0:1789760776.187267:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706291011463 time 706306485614 state 0x2 ref 0 00000100:00100000:0.0:1789760776.189357:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.189449:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767168 00000100:00100000:2.0:1789760776.189458:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767168 00000100:00100000:2.0:1789760776.189507:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706291011463 time 706306485614 state 0x0 ref 1 00000100:00100000:2.0:1789760776.189521:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706306746030 time 706306485614 state 0x1 ref 1 00000100:00100000:2.0:1789760776.189551:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706306746030 time 706314157707 state 0x0 ref 1 00000100:00100000:2.0:1789760776.189564:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.189616:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76144640, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24508 00010000:00010000:2.0:1789760776.189626:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.189640:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.189644:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.189650:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.189676:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.193520:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985897 is committed 00002000:00010000:2.0:1789760776.193639:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76144640 -> 76148736 00002000:00010000:2.0:1789760776.193644:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148744 -> 148752 00010000:00010000:2.0:1789760776.193656:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.193702:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.193712:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.193720:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.193725:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.193733:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.193748:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.193752:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f045b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76144640->76148735] (req 76144640->76148735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.193837:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4271us (4484us total) trans 4294985897 rc 0/0 00000100:00100000:2.0:1789760776.193847:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18601 00000100:00100000:2.0:1789760776.193857:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706306746030 time 706314157707 state 0x2 ref 0 00000100:00100000:1.0:1789760776.197703:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.197808:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767296 00000100:00100000:2.0:1789760776.197828:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767296 00000100:00100000:2.0:1789760776.197884:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706306746030 time 706314157707 state 0x0 ref 1 00000100:00100000:2.0:1789760776.197901:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706314418123 time 706314157707 state 0x1 ref 1 00000100:00100000:2.0:1789760776.197933:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706314418123 time 706322539035 state 0x0 ref 1 00000100:00100000:2.0:1789760776.197950:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.197994:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76148736, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24510 00010000:00010000:2.0:1789760776.198003:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.198017:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.198021:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.198026:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.198037:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.206638:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985898 is committed 00002000:00010000:2.0:1789760776.206747:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76148736 -> 76152832 00002000:00010000:2.0:1789760776.206752:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148752 -> 148760 00010000:00010000:2.0:1789760776.206765:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.206778:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.206787:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.206794:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.206799:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.206807:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.206827:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.206835:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0462 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76148736->76152831] (req 76148736->76152831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.206927:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8980us (9229us total) trans 4294985898 rc 0/0 00000100:00100000:2.0:1789760776.206937:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18602 00000100:00100000:2.0:1789760776.206947:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706314418123 time 706322539035 state 0x2 ref 0 00000100:00100000:3.0:1789760776.213131:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760776.213309:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767424 00000100:00100000:1.0:1789760776.213320:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767424 00000100:00100000:1.0:1789760776.213386:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706314418123 time 706322539035 state 0x0 ref 1 00000100:00100000:1.0:1789760776.213434:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706322799451 time 706322539035 state 0x1 ref 1 00000100:00100000:1.0:1789760776.213579:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706322799451 time 706338184042 state 0x0 ref 1 00000100:00100000:1.0:1789760776.213592:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503767424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.213650:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76152832, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24518 00010000:00010000:1.0:1789760776.213660:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.213682:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760776.213687:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.213693:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.213707:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.218363:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985899 is committed 00002000:00010000:1.0:1789760776.218510:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76152832 -> 76156928 00002000:00010000:1.0:1789760776.218515:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148760 -> 148768 00010000:00010000:1.0:1789760776.218529:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.218542:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.218552:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.218559:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.218675:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.218687:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.218713:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.218721:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0469 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76152832->76156927] (req 76152832->76156927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.218972:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503767424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5376us (5845us total) trans 4294985899 rc 0/0 00000100:00100000:1.0:1789760776.218983:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18603 00000100:00100000:1.0:1789760776.218993:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706322799451 time 706338184042 state 0x2 ref 0 00000100:00100000:3.0:1789760776.221532:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.221632:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767552 00000100:00100000:0.0:1789760776.221642:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767552 00000100:00100000:0.0:1789760776.221706:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706322799451 time 706338184042 state 0x0 ref 1 00000100:00100000:0.0:1789760776.221721:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706338444458 time 706338184042 state 0x1 ref 1 00000100:00100000:0.0:1789760776.221751:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706338444458 time 706346357963 state 0x0 ref 1 00000100:00100000:0.0:1789760776.221763:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.221808:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76156928, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24520 00010000:00010000:0.0:1789760776.221817:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.221832:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.221837:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.221843:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.221858:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.228502:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985900 is committed 00002000:00010000:0.0:1789760776.229012:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76156928 -> 76161024 00002000:00010000:0.0:1789760776.229018:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148768 -> 148776 00010000:00010000:0.0:1789760776.229032:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.229044:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.229063:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.229071:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.229076:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.229084:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.229102:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.229109:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0470 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76156928->76161023] (req 76156928->76161023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.229245:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7477us (7716us total) trans 4294985900 rc 0/0 00000100:00100000:0.0:1789760776.229255:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18604 00000100:00100000:0.0:1789760776.229266:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706338444458 time 706346357963 state 0x2 ref 0 00000100:00100000:2.0:1789760776.232357:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.233218:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767680 00000100:00100000:0.0:1789760776.233230:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767680 00000100:00100000:0.0:1789760776.233288:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706338444458 time 706346357963 state 0x0 ref 1 00000100:00100000:0.0:1789760776.233303:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706346618379 time 706346357963 state 0x1 ref 1 00000100:00100000:1.0:1789760776.234864:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706346618379 time 706359462536 state 0x0 ref 1 00000100:00100000:1.0:1789760776.234891:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503767680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.234942:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76161024, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24528 00010000:00010000:1.0:1789760776.234951:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.234967:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760776.234975:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.234982:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.234996:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.240587:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985901 is committed 00002000:00010000:1.0:1789760776.240852:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76161024 -> 76165120 00002000:00010000:1.0:1789760776.240858:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148776 -> 148784 00010000:00010000:1.0:1789760776.240872:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.240885:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.240894:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.240901:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.240906:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.240915:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.240937:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.240945:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0477 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76161024->76165119] (req 76161024->76165119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.241037:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503767680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6152us (8685us total) trans 4294985901 rc 0/0 00000100:00100000:1.0:1789760776.241057:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18605 00000100:00100000:1.0:1789760776.241068:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706346618379 time 706359462536 state 0x2 ref 0 00000100:00100000:3.0:1789760776.246041:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.249573:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767808 00000100:00100000:2.0:1789760776.249599:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767808 00000100:00100000:2.0:1789760776.249671:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706346618379 time 706359462536 state 0x0 ref 1 00000100:00100000:2.0:1789760776.249685:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706359722952 time 706359462536 state 0x1 ref 1 00000100:00100000:2.0:1789760776.249722:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706359722952 time 706374329835 state 0x0 ref 1 00000100:00100000:2.0:1789760776.249735:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.249781:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76165120, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24530 00010000:00010000:2.0:1789760776.249790:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.249805:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.249809:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.249815:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.249827:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.258699:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985902 is committed 00002000:00010000:2.0:1789760776.258814:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76165120 -> 76169216 00002000:00010000:2.0:1789760776.258820:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148784 -> 148792 00010000:00010000:2.0:1789760776.258832:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.258844:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.258853:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.258859:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.258864:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.258873:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.258892:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.258909:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f047e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76165120->76169215] (req 76165120->76169215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.259009:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503767808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9272us (12973us total) trans 4294985902 rc 0/0 00000100:00100000:2.0:1789760776.259019:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18606 00000100:00100000:2.0:1789760776.259030:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706359722952 time 706374329835 state 0x2 ref 0 00000100:00100000:3.0:1789760776.262271:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.268244:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503767936 00000100:00100000:0.0:1789760776.268259:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503767936 00000100:00100000:0.0:1789760776.268340:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706359722952 time 706374329835 state 0x0 ref 1 00000100:00100000:0.0:1789760776.268357:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706374590251 time 706374329835 state 0x1 ref 1 00000100:00100000:0.0:1789760776.268405:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706374590251 time 706393010859 state 0x0 ref 1 00000100:00100000:0.0:1789760776.268419:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503767936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.268468:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76169216, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24538 00010000:00010000:0.0:1789760776.268477:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.268494:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.268499:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.268506:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.268520:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.276574:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985903 is committed 00002000:00010000:3.0:1789760776.277052:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76169216 -> 76173312 00002000:00010000:3.0:1789760776.277057:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148792 -> 148800 00010000:00010000:3.0:1789760776.277070:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.277084:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.277094:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.277102:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.277108:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.277117:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.277139:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.277147:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0485 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76169216->76173311] (req 76169216->76173311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.277582:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503767936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9159us (15313us total) trans 4294985903 rc 0/0 00000100:00100000:3.0:1789760776.277594:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18607 00000100:00100000:3.0:1789760776.277604:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706374590251 time 706393010859 state 0x2 ref 0 00000100:00100000:1.0:1789760776.281118:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.281199:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768064 00000100:00100000:3.0:1789760776.281209:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768064 00000100:00100000:3.0:1789760776.281265:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706374590251 time 706393010859 state 0x0 ref 1 00000100:00100000:3.0:1789760776.281280:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706393271275 time 706393010859 state 0x1 ref 1 00000100:00100000:3.0:1789760776.281312:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706393271275 time 706405918966 state 0x0 ref 1 00000100:00100000:3.0:1789760776.281325:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.281385:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76173312, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24540 00010000:00010000:3.0:1789760776.281432:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.281448:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.281453:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.281460:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.281474:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.286114:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985904 is committed 00002000:00010000:0.0:1789760776.286304:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76173312 -> 76177408 00002000:00010000:0.0:1789760776.286309:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148800 -> 148808 00010000:00010000:0.0:1789760776.286321:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286334:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286342:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286349:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.286354:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286363:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286382:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.286427:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f048c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76173312->76177407] (req 76173312->76177407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.286605:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5276us (5491us total) trans 4294985904 rc 0/0 00000100:00100000:1.0:1789760776.286628:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18608 00000100:00100000:1.0:1789760776.286646:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706393271275 time 706405918966 state 0x2 ref 0 00000100:00100000:0.0:1789760776.289047:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760776.289161:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768192 00000100:00100000:1.0:1789760776.289171:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768192 00000100:00100000:1.0:1789760776.289224:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706393271275 time 706405918966 state 0x0 ref 1 00000100:00100000:1.0:1789760776.289239:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706406179382 time 706405918966 state 0x1 ref 1 00000100:00100000:1.0:1789760776.289316:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706406179382 time 706413921753 state 0x0 ref 1 00000100:00100000:1.0:1789760776.289329:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503768192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.289373:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76177408, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24548 00010000:00010000:1.0:1789760776.289382:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.289440:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760776.289445:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.289451:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.289469:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.298169:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985905 is committed 00002000:00010000:1.0:1789760776.298867:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76177408 -> 76181504 00002000:00010000:1.0:1789760776.298875:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148808 -> 148816 00010000:00010000:1.0:1789760776.298888:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.298901:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.298909:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.298915:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.298920:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.298930:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.298949:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.299068:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f0493 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76177408->76181503] (req 76177408->76181503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.299202:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503768192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9871us (10159us total) trans 4294985905 rc 0/0 00000100:00100000:1.0:1789760776.299213:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18609 00000100:00100000:1.0:1789760776.299222:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706406179382 time 706413921753 state 0x2 ref 0 00000100:00100000:3.0:1789760776.303302:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.303482:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768320 00000100:00100000:2.0:1789760776.303492:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768320 00000100:00100000:2.0:1789760776.303546:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706406179382 time 706413921753 state 0x0 ref 1 00000100:00100000:2.0:1789760776.303562:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706414182169 time 706413921753 state 0x1 ref 1 00000100:00100000:2.0:1789760776.304949:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706414182169 time 706429548478 state 0x0 ref 1 00000100:00100000:2.0:1789760776.304972:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503768320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.305019:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76181504, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24550 00010000:00010000:2.0:1789760776.305027:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.305045:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.305049:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.305055:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.305068:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.310566:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985906 is committed 00002000:00010000:2.0:1789760776.310777:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76181504 -> 76185600 00002000:00010000:2.0:1789760776.310782:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148816 -> 148824 00010000:00010000:2.0:1789760776.310795:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310928:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310941:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310949:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.310954:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310962:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310981:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310989:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f049a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76181504->76185599] (req 76181504->76185599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.311975:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503768320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7002us (8673us total) trans 4294985906 rc 0/0 00000100:00100000:2.0:1789760776.311987:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18610 00000100:00100000:2.0:1789760776.311997:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706414182169 time 706429548478 state 0x2 ref 0 00000100:00100000:2.0:1789760776.316475:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.316558:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768448 00000100:00100000:3.0:1789760776.316568:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768448 00000100:00100000:3.0:1789760776.316777:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706414182169 time 706429548478 state 0x0 ref 1 00000100:00100000:3.0:1789760776.316792:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706429808894 time 706429548478 state 0x1 ref 1 00000100:00100000:3.0:1789760776.316879:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706429808894 time 706441484910 state 0x0 ref 1 00000100:00100000:3.0:1789760776.316892:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503768448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.316938:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76185600, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24558 00010000:00010000:3.0:1789760776.316947:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316962:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.316967:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.316973:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316987:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.321512:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985907 is committed 00002000:00010000:3.0:1789760776.321827:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76185600 -> 76189696 00002000:00010000:3.0:1789760776.321833:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148824 -> 148832 00010000:00010000:3.0:1789760776.321846:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.321858:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.321866:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.321873:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.321877:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.321887:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.321967:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.321981:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76185600->76189695] (req 76185600->76189695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.322195:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503768448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5299us (5723us total) trans 4294985907 rc 0/0 00000100:00100000:3.0:1789760776.322206:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18611 00000100:00100000:3.0:1789760776.322215:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706429808894 time 706441484910 state 0x2 ref 0 00000100:00100000:1.0:1789760776.333458:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.333958:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768576 00000100:00100000:3.0:1789760776.333968:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768576 00000100:00100000:3.0:1789760776.334022:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706429808894 time 706441484910 state 0x0 ref 1 00000100:00100000:3.0:1789760776.334038:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706441745326 time 706441484910 state 0x1 ref 1 00000100:00100000:3.0:1789760776.334116:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706441745326 time 706458720997 state 0x0 ref 1 00000100:00100000:3.0:1789760776.334130:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.334411:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76189696, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24560 00010000:00010000:3.0:1789760776.334423:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.334439:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.334443:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.334449:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.334468:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.339441:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985908 is committed 00002000:00010000:3.0:1789760776.340518:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76189696 -> 76193792 00002000:00010000:3.0:1789760776.340526:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148832 -> 148840 00010000:00010000:3.0:1789760776.340538:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.340552:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.340700:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.340712:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.340717:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.340726:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.340746:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.340753:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76189696->76193791] (req 76189696->76193791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.340890:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6758us (7436us total) trans 4294985908 rc 0/0 00000100:00100000:3.0:1789760776.340899:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18612 00000100:00100000:3.0:1789760776.340908:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706441745326 time 706458720997 state 0x2 ref 0 00000100:00100000:1.0:1789760776.344809:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.346582:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768704 00000100:00100000:2.0:1789760776.346604:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768704 00000100:00100000:2.0:1789760776.346673:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706441745326 time 706458720997 state 0x0 ref 1 00000100:00100000:2.0:1789760776.346689:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706458981413 time 706458720997 state 0x1 ref 1 00000100:00100000:2.0:1789760776.346718:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706458981413 time 706471324435 state 0x0 ref 1 00000100:00100000:2.0:1789760776.346732:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.346787:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76193792, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24568 00010000:00010000:2.0:1789760776.346796:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.346812:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.346816:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.346822:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.346835:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.351259:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985909 is committed 00002000:00010000:2.0:1789760776.351359:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76193792 -> 76197888 00002000:00010000:2.0:1789760776.351364:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148840 -> 148848 00010000:00010000:2.0:1789760776.351377:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.351415:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.351424:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.351431:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.351435:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.351443:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.351462:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.351469:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76193792->76197887] (req 76193792->76197887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.351551:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4818us (6748us total) trans 4294985909 rc 0/0 00000100:00100000:2.0:1789760776.351560:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18613 00000100:00100000:2.0:1789760776.351576:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706458981413 time 706471324435 state 0x2 ref 0 00000100:00100000:2.0:1789760776.358258:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.359144:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768832 00000100:00100000:0.0:1789760776.359157:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768832 00000100:00100000:0.0:1789760776.359214:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706458981413 time 706471324435 state 0x0 ref 1 00000100:00100000:0.0:1789760776.359231:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706471584851 time 706471324435 state 0x1 ref 1 00000100:00100000:0.0:1789760776.359258:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706471584851 time 706483864551 state 0x0 ref 1 00000100:00100000:0.0:1789760776.359271:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.359320:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76197888, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24570 00010000:00010000:0.0:1789760776.359330:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.359355:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.359360:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.359366:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.359380:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.365464:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985910 is committed 00002000:00010000:0.0:1789760776.365749:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76197888 -> 76201984 00002000:00010000:0.0:1789760776.365755:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148848 -> 148856 00010000:00010000:0.0:1789760776.365769:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.365782:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.365793:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.365801:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.365806:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.365815:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.365836:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.365844:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f04b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76197888->76201983] (req 76197888->76201983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.366199:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503768832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6925us (7944us total) trans 4294985910 rc 0/0 00000100:00100000:0.0:1789760776.366210:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18614 00000100:00100000:0.0:1789760776.366221:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706471584851 time 706483864551 state 0x2 ref 0 00000100:00100000:1.0:1789760776.372309:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.372822:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503768960 00000100:00100000:3.0:1789760776.372833:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503768960 00000100:00100000:2.0:1789760776.373166:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706471584851 time 706483864551 state 0x0 ref 1 00000100:00100000:2.0:1789760776.373182:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706484124967 time 706483864551 state 0x1 ref 1 00000100:00100000:2.0:1789760776.373252:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706484124967 time 706497858126 state 0x0 ref 1 00000100:00100000:2.0:1789760776.373266:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503768960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.373316:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76201984, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24578 00010000:00010000:2.0:1789760776.373325:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.373352:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.373357:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.373363:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.373377:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.379062:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985911 is committed 00002000:00010000:2.0:1789760776.379283:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76201984 -> 76206080 00002000:00010000:2.0:1789760776.379289:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148856 -> 148864 00010000:00010000:2.0:1789760776.379303:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.379316:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.379325:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.379467:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.379472:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.379481:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.379501:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.379509:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76201984->76206079] (req 76201984->76206079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.379772:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503768960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6503us (7468us total) trans 4294985911 rc 0/0 00000100:00100000:2.0:1789760776.379782:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18615 00000100:00100000:2.0:1789760776.379792:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706484124967 time 706497858126 state 0x2 ref 0 00000100:00100000:1.0:1789760776.384831:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.385323:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769088 00000100:00100000:3.0:1789760776.385345:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769088 00000100:00100000:3.0:1789760776.385434:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706484124967 time 706497858126 state 0x0 ref 1 00000100:00100000:3.0:1789760776.385450:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706498118542 time 706497858126 state 0x1 ref 1 00000100:00100000:3.0:1789760776.385473:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706498118542 time 706510079954 state 0x0 ref 1 00000100:00100000:3.0:1789760776.385485:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503769088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.385541:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76206080, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24580 00010000:00010000:3.0:1789760776.385550:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.385566:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.385570:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.385576:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.385589:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.390577:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985912 is committed 00002000:00010000:3.0:1789760776.390778:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76206080 -> 76210176 00002000:00010000:3.0:1789760776.390784:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148864 -> 148872 00010000:00010000:3.0:1789760776.390796:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.390810:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.390819:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.390827:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.390831:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.390840:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.390859:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.390867:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76206080->76210175] (req 76206080->76210175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.391049:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503769088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5561us (6223us total) trans 4294985912 rc 0/0 00000100:00100000:3.0:1789760776.391061:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18616 00000100:00100000:3.0:1789760776.391071:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706498118542 time 706510079954 state 0x2 ref 0 00000100:00100000:1.0:1789760776.395074:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.395150:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769216 00000100:00100000:2.0:1789760776.395162:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769216 00000100:00100000:2.0:1789760776.395219:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706498118542 time 706510079954 state 0x0 ref 1 00000100:00100000:2.0:1789760776.395236:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706510340370 time 706510079954 state 0x1 ref 1 00000100:00100000:2.0:1789760776.395272:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706510340370 time 706519878022 state 0x0 ref 1 00000100:00100000:2.0:1789760776.395287:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503769216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.395332:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76210176, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24588 00010000:00010000:2.0:1789760776.395341:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.395356:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.395360:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.395366:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.395379:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.401125:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985913 is committed 00002000:00010000:2.0:1789760776.402182:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76210176 -> 76214272 00002000:00010000:2.0:1789760776.402192:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148872 -> 148880 00010000:00010000:2.0:1789760776.402207:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.402220:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.402230:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.402238:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.402243:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.402253:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.402272:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.402279:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f04cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76210176->76214271] (req 76210176->76214271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.402941:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6968e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523608064:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760776.408005:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503769216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12706us (12927us total) trans 4294985913 rc 0/0 00000100:00100000:3.0:1789760776.408024:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760776.408028:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18617 00000100:00100000:2.0:1789760776.408041:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706510340370 time 706519878022 state 0x2 ref 0 00000100:00100000:0.0:1789760776.408123:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608064 00000100:00100000:0.0:1789760776.408132:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608064 00000100:00100000:0.0:1789760776.408184:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:0.0:1789760776.408194:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523608064:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760776.408233:0:6517:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:0.0:1789760776.408304:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523608064:12345-0@lo:41:osp-pre-1-0.0 Request processed in 111us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760776.408311:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:3.0:1789760776.408348:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6968e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523608064:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760776.414154:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.414774:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769344 00000100:00100000:0.0:1789760776.414785:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769344 00000100:00100000:0.0:1789760776.414839:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706510340370 time 706519878022 state 0x0 ref 1 00000100:00100000:0.0:1789760776.414854:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706520138438 time 706519878022 state 0x1 ref 1 00000100:00100000:0.0:1789760776.414874:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706520138438 time 706539481430 state 0x0 ref 1 00000100:00100000:0.0:1789760776.414901:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503769344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.414959:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76214272, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24590 00010000:00010000:0.0:1789760776.414969:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.414985:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.414989:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.414995:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.415009:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.419760:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985914 is committed 00002000:00010000:0.0:1789760776.419951:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76214272 -> 76218368 00002000:00010000:0.0:1789760776.419957:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148880 -> 148888 00010000:00010000:0.0:1789760776.419978:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.419992:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.420000:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.420008:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.420013:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.420023:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.420044:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.420052:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76214272->76218367] (req 76214272->76218367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.420145:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503769344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5242us (5995us total) trans 4294985914 rc 0/0 00000100:00100000:0.0:1789760776.420155:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18618 00000100:00100000:0.0:1789760776.420166:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706520138438 time 706539481430 state 0x2 ref 0 00000100:00100000:3.0:1789760776.426776:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.427496:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769472 00000100:00100000:0.0:1789760776.427507:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769472 00000100:00100000:0.0:1789760776.427563:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706520138438 time 706539481430 state 0x0 ref 1 00000100:00100000:0.0:1789760776.427579:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706539741846 time 706539481430 state 0x1 ref 1 00000100:00100000:0.0:1789760776.427626:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706539741846 time 706552232044 state 0x0 ref 1 00000100:00100000:0.0:1789760776.427640:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503769472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.427698:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76218368, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24598 00010000:00010000:0.0:1789760776.427707:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.427722:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.427727:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.427734:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.427747:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.432887:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985915 is committed 00002000:00010000:3.0:1789760776.433002:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76218368 -> 76222464 00002000:00010000:3.0:1789760776.433007:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148888 -> 148896 00010000:00010000:3.0:1789760776.433019:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.433040:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.433048:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.433054:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.433059:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.433072:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.433096:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.433103:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f04d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76218368->76222463] (req 76218368->76222463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.433204:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503769472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5562us (6435us total) trans 4294985915 rc 0/0 00000100:00100000:3.0:1789760776.433215:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18619 00000100:00100000:3.0:1789760776.433224:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706539741846 time 706552232044 state 0x2 ref 0 00000100:00100000:1.0:1789760776.443075:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.443186:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769600 00000100:00100000:3.0:1789760776.443195:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769600 00000100:00100000:3.0:1789760776.443250:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706539741846 time 706552232044 state 0x0 ref 1 00000100:00100000:3.0:1789760776.443265:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706552492460 time 706552232044 state 0x1 ref 1 00000100:00100000:3.0:1789760776.443295:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706552492460 time 706567902391 state 0x0 ref 1 00000100:00100000:3.0:1789760776.443307:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503769600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.443485:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76222464, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245a0 00010000:00010000:3.0:1789760776.443494:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.443509:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.443514:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.443521:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.443535:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.451741:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985916 is committed 00002000:00010000:1.0:1789760776.454694:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76222464 -> 76226560 00002000:00010000:1.0:1789760776.454703:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148896 -> 148904 00010000:00010000:1.0:1789760776.454715:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.454733:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.454740:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.454747:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.454751:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.454759:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.454776:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.454783:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f04e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76222464->76226559] (req 76222464->76226559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.454872:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503769600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11563us (11802us total) trans 4294985916 rc 0/0 00000100:00100000:1.0:1789760776.454888:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18620 00000100:00100000:1.0:1789760776.455088:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706552492460 time 706567902391 state 0x2 ref 0 00000100:00100000:1.0:1789760776.464206:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.464324:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769728 00000100:00100000:3.0:1789760776.464342:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769728 00000100:00100000:3.0:1789760776.464430:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706552492460 time 706567902391 state 0x0 ref 1 00000100:00100000:3.0:1789760776.464445:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706568162807 time 706567902391 state 0x1 ref 1 00000100:00100000:3.0:1789760776.464467:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706568162807 time 706589074539 state 0x0 ref 1 00000100:00100000:3.0:1789760776.464480:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503769728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.464525:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76226560, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245a8 00010000:00010000:3.0:1789760776.464535:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.464548:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.464552:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.464559:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.464571:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.471821:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985917 is committed 00002000:00010000:2.0:1789760776.471939:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76226560 -> 76230656 00002000:00010000:2.0:1789760776.471945:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148904 -> 148912 00010000:00010000:2.0:1789760776.471958:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.471971:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.471980:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.471993:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.471998:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.472008:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.472028:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.472036:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f04e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76226560->76230655] (req 76226560->76230655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.472141:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503769728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7659us (7938us total) trans 4294985917 rc 0/0 00000100:00100000:2.0:1789760776.472152:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18621 00000100:00100000:2.0:1789760776.472162:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706568162807 time 706589074539 state 0x2 ref 0 00000100:00100000:3.0:1789760776.476293:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.476663:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769856 00000100:00100000:0.0:1789760776.476673:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769856 00000100:00100000:0.0:1789760776.476731:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706568162807 time 706589074539 state 0x0 ref 1 00000100:00100000:0.0:1789760776.476747:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706589334955 time 706589074539 state 0x1 ref 1 00000100:00100000:0.0:1789760776.476784:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706589334955 time 706601389892 state 0x0 ref 1 00000100:00100000:0.0:1789760776.476797:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503769856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.476846:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76230656, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245b0 00010000:00010000:0.0:1789760776.476854:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.476870:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.476874:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.476881:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.476895:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.480514:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985918 is committed 00002000:00010000:0.0:1789760776.480633:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76230656 -> 76234752 00002000:00010000:0.0:1789760776.480639:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148912 -> 148920 00010000:00010000:0.0:1789760776.480663:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.480677:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.480687:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.480699:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.480704:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.480713:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.480732:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.480739:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76230656->76234751] (req 76230656->76234751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.480833:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503769856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4034us (4545us total) trans 4294985918 rc 0/0 00000100:00100000:0.0:1789760776.480845:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18622 00000100:00100000:0.0:1789760776.480855:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706589334955 time 706601389892 state 0x2 ref 0 00000100:00100000:1.0:1789760776.484206:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.484722:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503769984 00000100:00100000:0.0:1789760776.484733:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503769984 00000100:00100000:0.0:1789760776.484786:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706589334955 time 706601389892 state 0x0 ref 1 00000100:00100000:0.0:1789760776.484801:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706601650308 time 706601389892 state 0x1 ref 1 00000100:00100000:0.0:1789760776.484874:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706601650308 time 706609479563 state 0x0 ref 1 00000100:00100000:0.0:1789760776.484890:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503769984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.484934:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76234752, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245b8 00010000:00010000:0.0:1789760776.484943:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.484957:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.484962:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.484968:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.484981:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.488888:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985919 is committed 00002000:00010000:0.0:1789760776.489000:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76234752 -> 76238848 00002000:00010000:0.0:1789760776.489005:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148920 -> 148928 00010000:00010000:0.0:1789760776.489018:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.489030:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.489039:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.489047:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.489053:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.489062:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.489083:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.489091:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f04f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76234752->76238847] (req 76234752->76238847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.489193:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503769984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4301us (4990us total) trans 4294985919 rc 0/0 00000100:00100000:0.0:1789760776.489205:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18623 00000100:00100000:0.0:1789760776.489214:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706601650308 time 706609479563 state 0x2 ref 0 00000100:00100000:1.0:1789760776.493722:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.493802:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503770112 00000100:00100000:0.0:1789760776.493814:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503770112 00000100:00100000:0.0:1789760776.493870:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706601650308 time 706609479563 state 0x0 ref 1 00000100:00100000:0.0:1789760776.493885:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706609739979 time 706609479563 state 0x1 ref 1 00000100:00100000:0.0:1789760776.493919:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706609739979 time 706618525319 state 0x0 ref 1 00000100:00100000:0.0:1789760776.493933:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503770112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.493981:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76238848, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245c0 00010000:00010000:0.0:1789760776.493991:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.494006:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.494011:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.494017:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.494032:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.497525:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985920 is committed 00002000:00010000:0.0:1789760776.497855:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76238848 -> 76242944 00002000:00010000:0.0:1789760776.497860:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148928 -> 148936 00010000:00010000:0.0:1789760776.497872:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.497885:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.497895:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.497970:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.497975:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.497986:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.498007:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.498015:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f04fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76238848->76242943] (req 76238848->76242943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.498147:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503770112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4212us (4431us total) trans 4294985920 rc 0/0 00000100:00100000:0.0:1789760776.498158:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18624 00000100:00100000:0.0:1789760776.498168:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706609739979 time 706618525319 state 0x2 ref 0 00000100:00100000:1.0:1789760776.501233:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.501496:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503770240 00000100:00100000:2.0:1789760776.501506:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503770240 00000100:00100000:2.0:1789760776.501561:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706609739979 time 706618525319 state 0x0 ref 1 00000100:00100000:2.0:1789760776.501577:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706618785735 time 706618525319 state 0x1 ref 1 00000100:00100000:2.0:1789760776.501640:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706618785735 time 706626246110 state 0x0 ref 1 00000100:00100000:2.0:1789760776.501666:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503770240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.501713:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76242944, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245c8 00010000:00010000:2.0:1789760776.501723:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.501738:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.501742:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.501748:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.501762:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.505431:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985921 is committed 00002000:00010000:2.0:1789760776.505604:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76242944 -> 76247040 00002000:00010000:2.0:1789760776.505621:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148936 -> 148944 00010000:00010000:2.0:1789760776.505634:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.505655:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.505664:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.505671:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.505676:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.505685:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.505703:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.505710:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0503 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76242944->76247039] (req 76242944->76247039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.505801:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503770240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4133us (4573us total) trans 4294985921 rc 0/0 00000100:00100000:2.0:1789760776.505811:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18625 00000100:00100000:2.0:1789760776.505821:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706618785735 time 706626246110 state 0x2 ref 0 00000100:00100000:1.0:1789760776.513866:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.513939:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503770368 00000100:00100000:2.0:1789760776.513949:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503770368 00000100:00100000:2.0:1789760776.514004:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706618785735 time 706626246110 state 0x0 ref 1 00000100:00100000:2.0:1789760776.514018:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706626506526 time 706626246110 state 0x1 ref 1 00000100:00100000:2.0:1789760776.514052:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706626506526 time 706638658058 state 0x0 ref 1 00000100:00100000:2.0:1789760776.514065:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503770368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.514112:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76247040, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245d0 00010000:00010000:2.0:1789760776.514122:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.514138:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.514143:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.514149:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.514164:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.522680:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985922 is committed 00002000:00010000:2.0:1789760776.522913:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76247040 -> 76251136 00002000:00010000:2.0:1789760776.522918:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148944 -> 148952 00010000:00010000:2.0:1789760776.522930:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.522942:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.522950:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.522958:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.522962:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.522972:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.522991:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.523012:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f050a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76247040->76251135] (req 76247040->76251135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.523103:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503770368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9035us (9242us total) trans 4294985922 rc 0/0 00000100:00100000:2.0:1789760776.523112:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18626 00000100:00100000:2.0:1789760776.523121:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706626506526 time 706638658058 state 0x2 ref 0 00000100:00100000:1.0:1789760776.525739:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.525831:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503770496 00000100:00100000:0.0:1789760776.525841:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503770496 00000100:00100000:0.0:1789760776.525897:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706626506526 time 706638658058 state 0x0 ref 1 00000100:00100000:0.0:1789760776.525910:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706638918474 time 706638658058 state 0x1 ref 1 00000100:00100000:0.0:1789760776.525936:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706638918474 time 706650542997 state 0x0 ref 1 00000100:00100000:0.0:1789760776.525948:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503770496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.526004:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76251136, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245d8 00010000:00010000:0.0:1789760776.526013:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.526028:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.526033:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.526039:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.526052:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.533345:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985923 is committed 00002000:00010000:0.0:1789760776.533515:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76251136 -> 76255232 00002000:00010000:0.0:1789760776.533520:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148952 -> 148960 00010000:00010000:0.0:1789760776.533532:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.533544:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.533553:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.533561:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.533565:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.533574:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.533592:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.533604:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0511 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76251136->76255231] (req 76251136->76255231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.533763:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503770496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7810us (8027us total) trans 4294985923 rc 0/0 00000100:00100000:0.0:1789760776.533774:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18627 00000100:00100000:0.0:1789760776.533784:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706638918474 time 706650542997 state 0x2 ref 0 00000100:00100000:1.0:1789760776.540714:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.541128:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503770624 00000100:00100000:0.0:1789760776.541139:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503770624 00000100:00100000:0.0:1789760776.541195:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706638918474 time 706650542997 state 0x0 ref 1 00000100:00100000:0.0:1789760776.541209:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706650803413 time 706650542997 state 0x1 ref 1 00000100:00100000:0.0:1789760776.541285:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706650803413 time 706665891276 state 0x0 ref 1 00000100:00100000:0.0:1789760776.541297:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503770624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.541344:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76255232, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245e0 00010000:00010000:0.0:1789760776.541353:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.541367:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.541371:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.541378:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.541428:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.556225:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985924 is committed 00002000:00010000:0.0:1789760776.556353:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76255232 -> 76259328 00002000:00010000:0.0:1789760776.556359:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148960 -> 148968 00010000:00010000:0.0:1789760776.556372:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.556386:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.556431:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.556438:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.556442:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.556451:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.556473:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.556482:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0518 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76255232->76259327] (req 76255232->76259327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.556671:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503770624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15369us (15962us total) trans 4294985924 rc 0/0 00000100:00100000:1.0:1789760776.556681:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18628 00000100:00100000:1.0:1789760776.556690:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706650803413 time 706665891276 state 0x2 ref 0 00000100:00100000:1.0:1789760776.562603:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.562968:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503770752 00000100:00100000:3.0:1789760776.562978:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503770752 00000100:00100000:3.0:1789760776.563045:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706650803413 time 706665891276 state 0x0 ref 1 00000100:00100000:3.0:1789760776.563059:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706666151692 time 706665891276 state 0x1 ref 1 00000100:00100000:3.0:1789760776.563091:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706666151692 time 706687697618 state 0x0 ref 1 00000100:00100000:3.0:1789760776.563102:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503770752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760776.563151:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76259328, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245e8 00010000:00010000:3.0:1789760776.563161:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.563176:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760776.563180:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.563186:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.563204:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.581545:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985925 is committed 00002000:00010000:2.0:1789760776.586523:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76259328 -> 76263424 00002000:00010000:2.0:1789760776.586534:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148968 -> 148976 00010000:00010000:2.0:1789760776.586547:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.586559:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.586567:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.586575:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.586580:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.586594:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.586625:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.586633:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f051f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76259328->76263423] (req 76259328->76263423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.586735:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503770752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 23629us (24137us total) trans 4294985925 rc 0/0 00000100:00100000:2.0:1789760776.586746:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18629 00000100:00100000:2.0:1789760776.586756:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706666151692 time 706687697618 state 0x2 ref 0 00000100:00100000:3.0:1789760776.592897:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.592994:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503770880 00000100:00100000:2.0:1789760776.593005:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503770880 00000100:00100000:2.0:1789760776.593057:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706666151692 time 706687697618 state 0x0 ref 1 00000100:00100000:2.0:1789760776.593072:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706687958034 time 706687697618 state 0x1 ref 1 00000100:00100000:2.0:1789760776.593103:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706687958034 time 706717709160 state 0x0 ref 1 00000100:00100000:2.0:1789760776.593114:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503770880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.593160:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76263424, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245f0 00010000:00010000:2.0:1789760776.593169:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.593183:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.593187:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.593193:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.593206:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.604823:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985926 is committed 00002000:00010000:2.0:1789760776.609560:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76263424 -> 76267520 00002000:00010000:2.0:1789760776.609569:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148976 -> 148984 00010000:00010000:2.0:1789760776.609582:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.609593:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.609601:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.609619:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.609624:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.609633:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.609656:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.609663:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0526 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76263424->76267519] (req 76263424->76267519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.609757:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503770880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16640us (16864us total) trans 4294985926 rc 0/0 00000100:00100000:2.0:1789760776.609765:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18630 00000100:00100000:2.0:1789760776.609775:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706687958034 time 706717709160 state 0x2 ref 0 00000100:00100000:1.0:1789760776.617844:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.617951:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771008 00000100:00100000:0.0:1789760776.617962:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771008 00000100:00100000:0.0:1789760776.618016:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706687958034 time 706717709160 state 0x0 ref 1 00000100:00100000:0.0:1789760776.618033:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706717969576 time 706717709160 state 0x1 ref 1 00000100:00100000:0.0:1789760776.618096:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706717969576 time 706742701968 state 0x0 ref 1 00000100:00100000:0.0:1789760776.618109:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.618156:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76267520, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x245f8 00010000:00010000:0.0:1789760776.618165:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.618181:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.618185:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.618191:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.618205:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.633499:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985927 is committed 00002000:00010000:0.0:1789760776.633634:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76267520 -> 76271616 00002000:00010000:0.0:1789760776.633639:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148984 -> 148992 00010000:00010000:0.0:1789760776.633652:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.633669:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.633678:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.633686:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.633691:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.633701:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.633720:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.633728:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f052d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76267520->76271615] (req 76267520->76271615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.633825:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15714us (15986us total) trans 4294985927 rc 0/0 00000100:00100000:0.0:1789760776.633835:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18631 00000100:00100000:0.0:1789760776.633845:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706717969576 time 706742701968 state 0x2 ref 0 00000100:00100000:1.0:1789760776.640478:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.640756:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771136 00000100:00100000:2.0:1789760776.640769:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771136 00000100:00100000:2.0:1789760776.640822:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706717969576 time 706742701968 state 0x0 ref 1 00000100:00100000:2.0:1789760776.640837:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706742962384 time 706742701968 state 0x1 ref 1 00000100:00100000:2.0:1789760776.640858:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706742962384 time 706765465995 state 0x0 ref 1 00000100:00100000:2.0:1789760776.640869:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503771136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.640915:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76271616, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24600 00010000:00010000:2.0:1789760776.640925:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640939:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.640944:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.640950:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640964:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.648174:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985928 is committed 00002000:00010000:2.0:1789760776.648461:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76271616 -> 76275712 00002000:00010000:2.0:1789760776.648466:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 148992 -> 149000 00010000:00010000:2.0:1789760776.648477:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.648489:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.648498:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.648505:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.648509:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.648519:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.648537:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.648551:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0534 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76271616->76275711] (req 76271616->76275711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.649501:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503771136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8624us (9023us total) trans 4294985928 rc 0/0 00000100:00100000:2.0:1789760776.649514:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18632 00000100:00100000:2.0:1789760776.649524:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706742962384 time 706765465995 state 0x2 ref 0 00000100:00100000:3.0:1789760776.653069:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.655578:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771264 00000100:00100000:2.0:1789760776.655593:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771264 00000100:00100000:2.0:1789760776.655728:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706742962384 time 706765465995 state 0x0 ref 1 00000100:00100000:2.0:1789760776.655743:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706765726411 time 706765465995 state 0x1 ref 1 00000100:00100000:2.0:1789760776.655822:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706765726411 time 706780428326 state 0x0 ref 1 00000100:00100000:2.0:1789760776.655835:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.655885:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76275712, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24608 00010000:00010000:2.0:1789760776.655894:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.655908:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.655913:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.655919:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.655938:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.662926:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985929 is committed 00002000:00010000:2.0:1789760776.663036:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76275712 -> 76279808 00002000:00010000:2.0:1789760776.663041:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149000 -> 149008 00010000:00010000:2.0:1789760776.663053:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.663066:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.663076:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.663083:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.663087:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.663097:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.663115:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.663122:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f053b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76275712->76279807] (req 76275712->76279807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.663488:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7649us (10424us total) trans 4294985929 rc 0/0 00000100:00100000:2.0:1789760776.663499:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18633 00000100:00100000:2.0:1789760776.663510:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706765726411 time 706780428326 state 0x2 ref 0 00000100:00100000:1.0:1789760776.666866:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.667620:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771392 00000100:00100000:0.0:1789760776.667631:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771392 00000100:00100000:0.0:1789760776.667697:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706765726411 time 706780428326 state 0x0 ref 1 00000100:00100000:0.0:1789760776.667713:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706780688742 time 706780428326 state 0x1 ref 1 00000100:00100000:0.0:1789760776.667742:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706780688742 time 706792348148 state 0x0 ref 1 00000100:00100000:0.0:1789760776.667754:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503771392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.667801:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76279808, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24610 00010000:00010000:0.0:1789760776.667811:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.667832:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.667836:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.667842:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.667856:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.679749:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985930 is committed 00002000:00010000:0.0:1789760776.680125:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76279808 -> 76283904 00002000:00010000:0.0:1789760776.680130:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149008 -> 149016 00010000:00010000:0.0:1789760776.680143:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680156:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680171:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680178:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.680183:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680191:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680209:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680217:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0542 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76279808->76283903] (req 76279808->76283903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.681800:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503771392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14037us (14932us total) trans 4294985930 rc 0/0 00000100:00100000:0.0:1789760776.681814:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18634 00000100:00100000:0.0:1789760776.681824:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706780688742 time 706792348148 state 0x2 ref 0 00000100:00100000:1.0:1789760776.687804:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.687896:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771520 00000100:00100000:0.0:1789760776.687906:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771520 00000100:00100000:0.0:1789760776.687962:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706780688742 time 706792348148 state 0x0 ref 1 00000100:00100000:0.0:1789760776.687977:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706792608564 time 706792348148 state 0x1 ref 1 00000100:00100000:0.0:1789760776.688062:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706792608564 time 706812668403 state 0x0 ref 1 00000100:00100000:0.0:1789760776.688077:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.688126:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76283904, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24618 00010000:00010000:0.0:1789760776.688135:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.688150:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.688155:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.688161:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.688177:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.696984:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985931 is committed 00002000:00010000:3.0:1789760776.698480:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76283904 -> 76288000 00002000:00010000:3.0:1789760776.698488:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149016 -> 149024 00010000:00010000:3.0:1789760776.698502:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.698513:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.698521:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.698528:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760776.698532:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.698540:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.698558:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.698566:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0549 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76283904->76287999] (req 76283904->76287999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.699081:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11000us (11280us total) trans 4294985931 rc 0/0 00000100:00100000:3.0:1789760776.699092:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18635 00000100:00100000:3.0:1789760776.699102:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706792608564 time 706812668403 state 0x2 ref 0 00000100:00100000:3.0:1789760776.710105:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.710197:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771648 00000100:00100000:2.0:1789760776.710208:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771648 00000100:00100000:2.0:1789760776.710265:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706792608564 time 706812668403 state 0x0 ref 1 00000100:00100000:2.0:1789760776.710281:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706812928819 time 706812668403 state 0x1 ref 1 00000100:00100000:2.0:1789760776.710311:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706812928819 time 706834917469 state 0x0 ref 1 00000100:00100000:2.0:1789760776.710324:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503771648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.710373:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76288000, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24620 00010000:00010000:2.0:1789760776.710383:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.710436:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.710441:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.710447:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.710460:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.721291:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985932 is committed 00002000:00010000:2.0:1789760776.721418:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76288000 -> 76292096 00002000:00010000:2.0:1789760776.721424:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149024 -> 149032 00010000:00010000:2.0:1789760776.721436:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.721447:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.721456:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.721463:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.721468:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.721476:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.721495:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.721502:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0550 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76288000->76292095] (req 76288000->76292095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.721587:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503771648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11262us (11488us total) trans 4294985932 rc 0/0 00000100:00100000:2.0:1789760776.721597:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18636 00000100:00100000:2.0:1789760776.721615:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706812928819 time 706834917469 state 0x2 ref 0 00010000:00010000:3.0:1789760776.725261:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760776.732013:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.732137:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771776 00000100:00100000:2.0:1789760776.732148:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771776 00000100:00100000:2.0:1789760776.732214:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706812928819 time 706834917469 state 0x0 ref 1 00000100:00100000:2.0:1789760776.732229:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706835177885 time 706834917469 state 0x1 ref 1 00000100:00100000:2.0:1789760776.732261:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706835177885 time 706856867038 state 0x0 ref 1 00000100:00100000:2.0:1789760776.732273:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.732319:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76292096, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24628 00010000:00010000:2.0:1789760776.732328:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.732344:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.732348:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.732355:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.732368:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.738320:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985933 is committed 00002000:00010000:2.0:1789760776.738629:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76292096 -> 76296192 00002000:00010000:2.0:1789760776.738635:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149032 -> 149040 00010000:00010000:2.0:1789760776.738703:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.738716:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.738724:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.738731:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.738736:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.738745:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.738765:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.738772:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0557 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76292096->76296191] (req 76292096->76296191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.738920:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6645us (6913us total) trans 4294985933 rc 0/0 00000100:00100000:2.0:1789760776.738930:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18637 00000100:00100000:2.0:1789760776.738939:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706835177885 time 706856867038 state 0x2 ref 0 00000100:00100000:3.0:1789760776.748177:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.748266:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503771904 00000100:00100000:0.0:1789760776.748277:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503771904 00000100:00100000:0.0:1789760776.748342:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706835177885 time 706856867038 state 0x0 ref 1 00000100:00100000:0.0:1789760776.748356:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706857127454 time 706856867038 state 0x1 ref 1 00000100:00100000:0.0:1789760776.748378:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706857127454 time 706872985379 state 0x0 ref 1 00000100:00100000:0.0:1789760776.748436:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.748483:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76296192, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24630 00010000:00010000:0.0:1789760776.748491:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.748505:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.748510:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.748517:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.748530:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.759107:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985934 is committed 00002000:00010000:0.0:1789760776.759834:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76296192 -> 76300288 00002000:00010000:0.0:1789760776.759841:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149040 -> 149048 00010000:00010000:0.0:1789760776.759853:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.759865:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.759874:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.759892:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.759903:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.759912:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.759929:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.759936:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f055e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76296192->76300287] (req 76296192->76300287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.760037:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503771904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11642us (11866us total) trans 4294985934 rc 0/0 00000100:00100000:0.0:1789760776.760047:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18638 00000100:00100000:0.0:1789760776.760056:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706857127454 time 706872985379 state 0x2 ref 0 00000100:00100000:1.0:1789760776.766265:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.766551:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772032 00000100:00100000:3.0:1789760776.766731:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772032 00000100:00100000:2.0:1789760776.767076:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706857127454 time 706872985379 state 0x0 ref 1 00000100:00100000:2.0:1789760776.767095:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706873245795 time 706872985379 state 0x1 ref 1 00000100:00100000:2.0:1789760776.767163:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706873245795 time 706891769660 state 0x0 ref 1 00000100:00100000:2.0:1789760776.767176:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503772032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.767220:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76300288, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24638 00010000:00010000:2.0:1789760776.767232:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.767248:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.767253:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.767258:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.767272:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.773482:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985935 is committed 00002000:00010000:2.0:1789760776.774747:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76300288 -> 76304384 00002000:00010000:2.0:1789760776.774754:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149048 -> 149056 00010000:00010000:2.0:1789760776.774767:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.774779:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.774788:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.774805:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.774810:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.774820:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.774838:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.774846:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0565 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76300288->76304383] (req 76300288->76304383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.775347:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503772032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8168us (9085us total) trans 4294985935 rc 0/0 00000100:00100000:2.0:1789760776.775358:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18639 00000100:00100000:2.0:1789760776.775375:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706873245795 time 706891769660 state 0x2 ref 0 00000100:00100000:1.0:1789760776.784215:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.784572:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772160 00000100:00100000:2.0:1789760776.784582:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772160 00000100:00100000:2.0:1789760776.784656:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706873245795 time 706891769660 state 0x0 ref 1 00000100:00100000:2.0:1789760776.784671:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706892030076 time 706891769660 state 0x1 ref 1 00000100:00100000:2.0:1789760776.784703:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706892030076 time 706909310224 state 0x0 ref 1 00000100:00100000:2.0:1789760776.784716:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503772160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.784760:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76304384, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x24640 00010000:00010000:2.0:1789760776.784769:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.784783:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.784788:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.784794:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.784809:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.793308:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985936 is committed 00002000:00010000:2.0:1789760776.793558:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76304384 -> 76308480 00002000:00010000:2.0:1789760776.793564:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:2.0:1789760776.793566:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00002000:00010000:2.0:1789760776.793568:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149056 -> 149064 00010000:00010000:2.0:1789760776.793580:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.793591:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.793601:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.793619:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.793768:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.793780:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.793800:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.793807:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f056c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76304384->76308479] (req 76304384->76308479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.793988:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503772160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9270us (9780us total) trans 4294985936 rc 0/0 00000100:00100000:2.0:1789760776.793998:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18640 00000100:00100000:2.0:1789760776.794007:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706892030076 time 706909310224 state 0x2 ref 0 00000100:00100000:1.0:1789760776.797558:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.798044:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772288 00000100:00100000:0.0:1789760776.798055:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772288 00000100:00100000:0.0:1789760776.798110:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706892030076 time 706909310224 state 0x0 ref 1 00000100:00100000:0.0:1789760776.798124:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706909570640 time 706909310224 state 0x1 ref 1 00000100:00100000:0.0:1789760776.798202:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706909570640 time 706922807306 state 0x0 ref 1 00000100:00100000:0.0:1789760776.798215:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.798265:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76308480, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24648 00010000:00010000:0.0:1789760776.798274:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.798290:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.798294:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.798300:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.798313:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.803438:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985937 is committed 00002000:00010000:0.0:1789760776.803541:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76308480 -> 76312576 00002000:00010000:0.0:1789760776.803546:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149064 -> 149072 00010000:00010000:0.0:1789760776.803560:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.803573:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.803583:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.803590:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.803596:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.803621:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.803641:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.803649:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0573 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76308480->76312575] (req 76308480->76312575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.803836:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5618us (6282us total) trans 4294985937 rc 0/0 00000100:00100000:0.0:1789760776.803847:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18641 00000100:00100000:0.0:1789760776.803858:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706909570640 time 706922807306 state 0x2 ref 0 00000100:00100000:1.0:1789760776.818636:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.821487:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772416 00000100:00100000:0.0:1789760776.821501:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772416 00000100:00100000:0.0:1789760776.821568:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706909570640 time 706922807306 state 0x0 ref 1 00000100:00100000:0.0:1789760776.821583:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706923067722 time 706922807306 state 0x1 ref 1 00000100:00100000:0.0:1789760776.821629:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706923067722 time 706946235446 state 0x0 ref 1 00000100:00100000:0.0:1789760776.821644:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.821709:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76312576, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24650 00010000:00010000:0.0:1789760776.821727:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.821743:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.821747:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.821754:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.821767:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.827152:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985938 is committed 00002000:00010000:0.0:1789760776.829597:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76312576 -> 76316672 00002000:00010000:0.0:1789760776.829615:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149072 -> 149080 00010000:00010000:0.0:1789760776.829629:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.829697:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.829708:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.829716:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.829721:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.829730:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.829755:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.829764:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f057a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76312576->76316671] (req 76312576->76316671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.830149:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8503us (11516us total) trans 4294985938 rc 0/0 00000100:00100000:0.0:1789760776.830159:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18642 00000100:00100000:0.0:1789760776.830169:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706923067722 time 706946235446 state 0x2 ref 0 00000100:00100000:1.0:1789760776.834727:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.834851:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772544 00000100:00100000:2.0:1789760776.834860:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772544 00000100:00100000:2.0:1789760776.834903:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706923067722 time 706946235446 state 0x0 ref 1 00000100:00100000:2.0:1789760776.834914:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706946495862 time 706946235446 state 0x1 ref 1 00000100:00100000:2.0:1789760776.834937:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706946495862 time 706959545519 state 0x0 ref 1 00000100:00100000:2.0:1789760776.834949:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503772544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.834985:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76316672, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24658 00010000:00010000:2.0:1789760776.834991:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.835004:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.835007:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.835012:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.835021:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.839673:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985939 is committed 00002000:00010000:1.0:1789760776.839853:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76316672 -> 76320768 00002000:00010000:1.0:1789760776.839858:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149080 -> 149088 00010000:00010000:1.0:1789760776.839870:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839882:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839891:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839897:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.839902:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839911:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839930:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839937:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0581 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76316672->76320767] (req 76316672->76320767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.840025:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503772544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5071us (5301us total) trans 4294985939 rc 0/0 00000100:00100000:1.0:1789760776.840036:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18643 00000100:00100000:1.0:1789760776.840045:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706946495862 time 706959545519 state 0x2 ref 0 00000100:00100000:1.0:1789760776.844574:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.844674:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772672 00000100:00100000:2.0:1789760776.844684:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772672 00000100:00100000:2.0:1789760776.844734:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706946495862 time 706959545519 state 0x0 ref 1 00000100:00100000:2.0:1789760776.844745:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706959805935 time 706959545519 state 0x1 ref 1 00000100:00100000:2.0:1789760776.844769:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706959805935 time 706969375278 state 0x0 ref 1 00000100:00100000:2.0:1789760776.844781:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.844828:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76320768, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24660 00010000:00010000:2.0:1789760776.844837:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844858:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760776.844863:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.844869:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844882:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.849890:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985940 is committed 00002000:00010000:2.0:1789760776.850008:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76320768 -> 76324864 00002000:00010000:2.0:1789760776.850013:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149088 -> 149096 00010000:00010000:2.0:1789760776.850025:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.850038:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.850047:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.850055:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.850075:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.850085:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.850112:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.850127:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0588 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76320768->76324863] (req 76320768->76324863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.850215:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5432us (5687us total) trans 4294985940 rc 0/0 00000100:00100000:2.0:1789760776.850234:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18644 00000100:00100000:2.0:1789760776.850243:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706959805935 time 706969375278 state 0x2 ref 0 00000100:00100000:2.0:1789760776.855436:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.855514:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772800 00000100:00100000:0.0:1789760776.855777:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772800 00000100:00100000:0.0:1789760776.855836:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706959805935 time 706969375278 state 0x0 ref 1 00000100:00100000:0.0:1789760776.855851:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706969635694 time 706969375278 state 0x1 ref 1 00000100:00100000:0.0:1789760776.855911:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706969635694 time 706980517615 state 0x0 ref 1 00000100:00100000:0.0:1789760776.855923:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.855968:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76324864, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24668 00010000:00010000:0.0:1789760776.855977:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.855997:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.856001:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.856007:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.856020:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.861302:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985941 is committed 00002000:00010000:0.0:1789760776.861435:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76324864 -> 76328960 00002000:00010000:0.0:1789760776.861441:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149096 -> 149104 00010000:00010000:0.0:1789760776.861455:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.861467:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.861475:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.861481:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.861486:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.861495:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.861518:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.861750:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f058f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76324864->76328959] (req 76324864->76328959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.861874:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503772800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5948us (6492us total) trans 4294985941 rc 0/0 00000100:00100000:0.0:1789760776.861892:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18645 00000100:00100000:0.0:1789760776.861902:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706969635694 time 706980517615 state 0x2 ref 0 00000100:00100000:1.0:1789760776.867457:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.867568:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503772928 00000100:00100000:0.0:1789760776.867577:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503772928 00000100:00100000:0.0:1789760776.867650:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706969635694 time 706980517615 state 0x0 ref 1 00000100:00100000:0.0:1789760776.867674:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706980778031 time 706980517615 state 0x1 ref 1 00000100:00100000:0.0:1789760776.867704:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706980778031 time 706992310512 state 0x0 ref 1 00000100:00100000:0.0:1789760776.867715:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503772928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.867764:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76328960, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24670 00010000:00010000:0.0:1789760776.867773:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.867788:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760776.867793:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.867800:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.867814:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.874287:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985942 is committed 00002000:00010000:0.0:1789760776.875074:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76328960 -> 76333056 00002000:00010000:0.0:1789760776.875081:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149104 -> 149112 00010000:00010000:0.0:1789760776.875094:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.875107:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.875116:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.875124:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.875128:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.875136:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.875155:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.875169:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0596 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76328960->76333055] (req 76328960->76333055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.875259:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503772928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7540us (7804us total) trans 4294985942 rc 0/0 00000100:00100000:0.0:1789760776.875269:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18646 00000100:00100000:0.0:1789760776.875279:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706980778031 time 706992310512 state 0x2 ref 0 00000100:00100000:1.0:1789760776.879796:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.879878:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773056 00000100:00100000:2.0:1789760776.879888:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773056 00000100:00100000:2.0:1789760776.879942:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706980778031 time 706992310512 state 0x0 ref 1 00000100:00100000:2.0:1789760776.879958:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706992570928 time 706992310512 state 0x1 ref 1 00000100:00100000:2.0:1789760776.880023:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706992570928 time 707004629199 state 0x0 ref 1 00000100:00100000:2.0:1789760776.880037:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503773056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.880086:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76333056, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24678 00010000:00010000:2.0:1789760776.880096:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880112:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760776.880116:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.880123:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880138:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.885181:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985943 is committed 00002000:00010000:1.0:1789760776.885486:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76333056 -> 76337152 00002000:00010000:1.0:1789760776.885492:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149112 -> 149120 00010000:00010000:1.0:1789760776.885507:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.885521:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.885530:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.885537:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.885542:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.885552:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.885571:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.885584:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f059d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76333056->76337151] (req 76333056->76337151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.886240:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503773056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6199us (6579us total) trans 4294985943 rc 0/0 00000100:00100000:1.0:1789760776.886251:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18647 00000100:00100000:1.0:1789760776.886261:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706992570928 time 707004629199 state 0x2 ref 0 00000100:00100000:2.0:1789760776.889981:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760776.890077:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773184 00000100:00100000:1.0:1789760776.890087:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773184 00000100:00100000:1.0:1789760776.890139:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 706992570928 time 707004629199 state 0x0 ref 1 00000100:00100000:1.0:1789760776.890154:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707004889615 time 707004629199 state 0x1 ref 1 00000100:00100000:1.0:1789760776.890232:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707004889615 time 707014837632 state 0x0 ref 1 00000100:00100000:1.0:1789760776.890245:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc16050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503773184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.890290:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76337152, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24680 00010000:00010000:1.0:1789760776.890299:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.890314:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760776.890319:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.890324:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.890355:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.897100:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985944 is committed 00002000:00010000:1.0:1789760776.897999:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76337152 -> 76341248 00002000:00010000:1.0:1789760776.898006:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149120 -> 149128 00010000:00010000:1.0:1789760776.898019:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.898032:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.898041:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.898047:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.898052:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.898061:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.898079:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.898086:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76337152->76341247] (req 76337152->76341247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.898231:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc16050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503773184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7983us (8254us total) trans 4294985944 rc 0/0 00000100:00100000:1.0:1789760776.898241:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18648 00000100:00100000:1.0:1789760776.898251:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707004889615 time 707014837632 state 0x2 ref 0 00000100:00100000:3.0:1789760776.901245:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.901352:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773312 00000100:00100000:0.0:1789760776.901363:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773312 00000100:00100000:0.0:1789760776.901470:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707004889615 time 707014837632 state 0x0 ref 1 00000100:00100000:0.0:1789760776.901485:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707015098048 time 707014837632 state 0x1 ref 1 00000100:00100000:0.0:1789760776.901510:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707015098048 time 707026117201 state 0x0 ref 1 00000100:00100000:0.0:1789760776.901523:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503773312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.901567:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76341248, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24688 00010000:00010000:0.0:1789760776.901576:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.901588:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760776.901592:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.901600:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.901628:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.906275:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985945 is committed 00002000:00010000:0.0:1789760776.907789:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76341248 -> 76345344 00002000:00010000:0.0:1789760776.907796:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149128 -> 149136 00010000:00010000:0.0:1789760776.907811:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.907823:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.907832:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.907839:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.907843:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.907852:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.907872:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.907886:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76341248->76345343] (req 76341248->76345343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.907982:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503773312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6455us (6741us total) trans 4294985945 rc 0/0 00000100:00100000:0.0:1789760776.907998:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18649 00000100:00100000:0.0:1789760776.908008:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707015098048 time 707026117201 state 0x2 ref 0 00000100:00100000:1.0:1789760776.914149:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760776.914559:0:3637:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760776.914580:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523608192:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760776.914650:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760776.914864:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608192 00000100:00100000:1.0:1789760776.914872:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608192 00000100:00100000:1.0:1789760776.914926:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:1.0:1789760776.914936:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523608192:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760776.914967:0:6518:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:1.0:1789760776.915020:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523608192:12345-0@lo:41:osp-pre-0-1.0 Request processed in 85us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760776.915027:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:3.0:1789760776.915217:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523608192:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760776.915615:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773440 00000100:00100000:0.0:1789760776.915621:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773440 00000100:00100000:0.0:1789760776.915726:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707015098048 time 707026117201 state 0x0 ref 1 00000100:00100000:0.0:1789760776.915740:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707026377617 time 707026117201 state 0x1 ref 1 00000100:00100000:0.0:1789760776.915793:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707026377617 time 707040399582 state 0x0 ref 1 00000100:00100000:0.0:1789760776.915803:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503773440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.915841:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76345344, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24690 00010000:00010000:0.0:1789760776.915849:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.915865:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760776.915870:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.915877:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.915891:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.921569:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985946 is committed 00002000:00010000:1.0:1789760776.921727:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76345344 -> 76349440 00002000:00010000:1.0:1789760776.921732:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149136 -> 149144 00010000:00010000:1.0:1789760776.921744:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.921755:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.921763:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.921774:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.921779:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.921788:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.921806:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.921813:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f05b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76345344->76349439] (req 76345344->76349439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.922035:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503773440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6227us (7890us total) trans 4294985946 rc 0/0 00000100:00100000:1.0:1789760776.922046:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18650 00000100:00100000:1.0:1789760776.922056:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707026377617 time 707040399582 state 0x2 ref 0 00000100:00100000:2.0:1789760776.925104:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760776.925185:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773568 00000100:00100000:1.0:1789760776.925195:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773568 00000100:00100000:1.0:1789760776.925249:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707026377617 time 707040399582 state 0x0 ref 1 00000100:00100000:1.0:1789760776.925264:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707040659998 time 707040399582 state 0x1 ref 1 00000100:00100000:1.0:1789760776.925339:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707040659998 time 707049945376 state 0x0 ref 1 00000100:00100000:1.0:1789760776.925356:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503773568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.925426:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76349440, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24698 00010000:00010000:1.0:1789760776.925435:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.925451:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760776.925455:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.925461:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.925474:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.930108:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985947 is committed 00002000:00010000:1.0:1789760776.930375:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76349440 -> 76353536 00002000:00010000:1.0:1789760776.930381:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149144 -> 149152 00010000:00010000:1.0:1789760776.930504:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930518:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930528:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930535:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.930541:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930551:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930570:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930578:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76349440->76353535] (req 76349440->76353535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.931009:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503773568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5649us (5907us total) trans 4294985947 rc 0/0 00000100:00100000:1.0:1789760776.931021:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18651 00000100:00100000:1.0:1789760776.931030:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707040659998 time 707049945376 state 0x2 ref 0 00000100:00100000:3.0:1789760776.935950:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760776.937294:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773696 00000100:00100000:1.0:1789760776.937306:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773696 00000100:00100000:1.0:1789760776.937376:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707040659998 time 707049945376 state 0x0 ref 1 00000100:00100000:1.0:1789760776.937411:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707050205792 time 707049945376 state 0x1 ref 1 00000100:00100000:1.0:1789760776.937503:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707050205792 time 707062107796 state 0x0 ref 1 00000100:00100000:1.0:1789760776.937527:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503773696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760776.937574:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76353536, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246a0 00010000:00010000:1.0:1789760776.937582:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.937596:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760776.937601:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.937617:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.937631:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.943804:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985948 is committed 00002000:00010000:2.0:1789760776.944223:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76353536 -> 76357632 00002000:00010000:2.0:1789760776.944227:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149152 -> 149160 00010000:00010000:2.0:1789760776.944241:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944254:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944263:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944271:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.944276:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944286:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944305:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944313:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f05c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76353536->76357631] (req 76353536->76357631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.944483:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503773696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6955us (8539us total) trans 4294985948 rc 0/0 00000100:00100000:2.0:1789760776.944494:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18652 00000100:00100000:2.0:1789760776.944504:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707050205792 time 707062107796 state 0x2 ref 0 00000100:00100000:3.0:1789760776.948229:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760776.949961:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773824 00000100:00100000:0.0:1789760776.949973:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773824 00000100:00100000:0.0:1789760776.950030:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707050205792 time 707062107796 state 0x0 ref 1 00000100:00100000:0.0:1789760776.950044:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707062368212 time 707062107796 state 0x1 ref 1 00000100:00100000:0.0:1789760776.950073:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707062368212 time 707074679614 state 0x0 ref 1 00000100:00100000:0.0:1789760776.950084:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503773824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760776.950128:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76357632, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246a8 00010000:00010000:0.0:1789760776.950137:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.950151:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760776.950155:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.950160:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.950172:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.955579:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985949 is committed 00002000:00010000:0.0:1789760776.955759:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76357632 -> 76361728 00002000:00010000:0.0:1789760776.955766:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149160 -> 149168 00010000:00010000:0.0:1789760776.955779:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.955792:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.955802:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.955809:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760776.955814:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.955824:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.955843:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.955850:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76357632->76361727] (req 76357632->76361727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.955986:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503773824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5899us (7762us total) trans 4294985949 rc 0/0 00000100:00100000:0.0:1789760776.955998:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18653 00000100:00100000:0.0:1789760776.956008:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707062368212 time 707074679614 state 0x2 ref 0 00000100:00100000:1.0:1789760776.962298:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.962447:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503773952 00000100:00100000:2.0:1789760776.962457:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503773952 00000100:00100000:2.0:1789760776.962508:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707062368212 time 707074679614 state 0x0 ref 1 00000100:00100000:2.0:1789760776.962523:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707074940030 time 707074679614 state 0x1 ref 1 00000100:00100000:2.0:1789760776.962555:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707074940030 time 707087161782 state 0x0 ref 1 00000100:00100000:2.0:1789760776.962567:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503773952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.962624:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76361728, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246b0 00010000:00010000:2.0:1789760776.962635:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.962651:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760776.962656:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.962663:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.962677:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.968508:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985950 is committed 00002000:00010000:1.0:1789760776.968774:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76361728 -> 76365824 00002000:00010000:1.0:1789760776.968780:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149168 -> 149176 00010000:00010000:1.0:1789760776.968794:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.968806:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.968814:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.968821:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760776.968825:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.968834:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.968853:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.968860:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f05ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76361728->76365823] (req 76361728->76365823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.969163:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503773952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6592us (6868us total) trans 4294985950 rc 0/0 00000100:00100000:1.0:1789760776.969174:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18654 00000100:00100000:1.0:1789760776.969183:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707074940030 time 707087161782 state 0x2 ref 0 00000100:00100000:3.0:1789760776.974063:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.974191:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774080 00000100:00100000:2.0:1789760776.974202:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774080 00000100:00100000:2.0:1789760776.974252:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707074940030 time 707087161782 state 0x0 ref 1 00000100:00100000:2.0:1789760776.974266:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707087422198 time 707087161782 state 0x1 ref 1 00000100:00100000:2.0:1789760776.974299:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707087422198 time 707098905888 state 0x0 ref 1 00000100:00100000:2.0:1789760776.974312:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.974370:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76365824, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246b8 00010000:00010000:2.0:1789760776.974380:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.974438:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760776.974443:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.974450:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.974465:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.981638:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985951 is committed 00002000:00010000:2.0:1789760776.982230:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76365824 -> 76369920 00002000:00010000:2.0:1789760776.982236:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149176 -> 149184 00010000:00010000:2.0:1789760776.982251:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.982265:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.982275:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.982283:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760776.982288:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.982297:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.982318:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.983189:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76365824->76369919] (req 76365824->76369919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.983296:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8981us (9247us total) trans 4294985951 rc 0/0 00000100:00100000:2.0:1789760776.983306:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18655 00000100:00100000:2.0:1789760776.983317:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707087422198 time 707098905888 state 0x2 ref 0 00000100:00100000:3.0:1789760776.992330:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760776.992434:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774208 00000100:00100000:2.0:1789760776.992444:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774208 00000100:00100000:2.0:1789760776.992498:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707087422198 time 707098905888 state 0x0 ref 1 00000100:00100000:2.0:1789760776.992513:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707099166304 time 707098905888 state 0x1 ref 1 00000100:00100000:2.0:1789760776.992547:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707099166304 time 707117153341 state 0x0 ref 1 00000100:00100000:2.0:1789760776.992559:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760776.992613:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76369920, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246c0 00010000:00010000:2.0:1789760776.992622:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.992639:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760776.992643:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.992650:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.992670:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.000313:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985952 is committed 00002000:00010000:3.0:1789760777.000488:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76369920 -> 76374016 00002000:00010000:3.0:1789760777.000493:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149184 -> 149192 00010000:00010000:3.0:1789760777.000507:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.000520:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.000529:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.000535:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.000539:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.000548:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.000567:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.000575:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f05dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76369920->76374015] (req 76369920->76374015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.000681:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8119us (8493us total) trans 4294985952 rc 0/0 00000100:00100000:3.0:1789760777.000691:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18656 00000100:00100000:3.0:1789760777.000701:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707099166304 time 707117153341 state 0x2 ref 0 00000100:00100000:1.0:1789760777.004434:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.004526:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774336 00000100:00100000:0.0:1789760777.004536:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774336 00000100:00100000:0.0:1789760777.004585:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707099166304 time 707117153341 state 0x0 ref 1 00000100:00100000:0.0:1789760777.004599:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707117413757 time 707117153341 state 0x1 ref 1 00000100:00100000:0.0:1789760777.004637:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707117413757 time 707129244204 state 0x0 ref 1 00000100:00100000:0.0:1789760777.004648:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.004690:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76374016, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246c8 00010000:00010000:0.0:1789760777.004700:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.004715:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.004719:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.004724:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.004737:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.010490:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985953 is committed 00002000:00010000:0.0:1789760777.010629:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76374016 -> 76378112 00002000:00010000:0.0:1789760777.010635:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149192 -> 149200 00010000:00010000:0.0:1789760777.010647:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010660:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010675:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010682:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.010688:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010698:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010719:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010727:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76374016->76378111] (req 76374016->76378111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.010856:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6203us (6425us total) trans 4294985953 rc 0/0 00000100:00100000:0.0:1789760777.010867:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18657 00000100:00100000:0.0:1789760777.010877:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707117413757 time 707129244204 state 0x2 ref 0 00000100:00100000:1.0:1789760777.017758:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.017851:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774464 00000100:00100000:2.0:1789760777.017863:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774464 00000100:00100000:2.0:1789760777.017917:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707117413757 time 707129244204 state 0x0 ref 1 00000100:00100000:2.0:1789760777.017933:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707129504620 time 707129244204 state 0x1 ref 1 00000100:00100000:2.0:1789760777.017965:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707129504620 time 707142571657 state 0x0 ref 1 00000100:00100000:2.0:1789760777.017977:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503774464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.018025:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76378112, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246d0 00010000:00010000:2.0:1789760777.018034:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.018050:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.018055:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.018061:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.018074:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.024521:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985954 is committed 00002000:00010000:2.0:1789760777.024671:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76378112 -> 76382208 00002000:00010000:2.0:1789760777.024677:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149200 -> 149208 00010000:00010000:2.0:1789760777.024690:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.024703:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.024713:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.024721:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.024725:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.024734:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.024755:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.024763:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76378112->76382207] (req 76378112->76382207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.024854:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503774464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6873us (7100us total) trans 4294985954 rc 0/0 00000100:00100000:2.0:1789760777.024864:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18658 00000100:00100000:2.0:1789760777.024874:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707129504620 time 707142571657 state 0x2 ref 0 00000100:00100000:3.0:1789760777.030537:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.030637:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774592 00000100:00100000:2.0:1789760777.030647:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774592 00000100:00100000:2.0:1789760777.030701:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707129504620 time 707142571657 state 0x0 ref 1 00000100:00100000:2.0:1789760777.030718:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707142832073 time 707142571657 state 0x1 ref 1 00000100:00100000:2.0:1789760777.030751:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707142832073 time 707155357333 state 0x0 ref 1 00000100:00100000:2.0:1789760777.030763:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.030811:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76382208, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246d8 00010000:00010000:2.0:1789760777.030820:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030836:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.030841:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.030847:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030864:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.035491:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985955 is committed 00002000:00010000:2.0:1789760777.035684:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76382208 -> 76386304 00002000:00010000:2.0:1789760777.035690:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149208 -> 149216 00010000:00010000:2.0:1789760777.035704:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.035716:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.035725:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.035732:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.035737:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.035746:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.035767:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.035775:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f05f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76382208->76386303] (req 76382208->76386303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.035904:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5136us (5371us total) trans 4294985955 rc 0/0 00000100:00100000:2.0:1789760777.035915:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18659 00000100:00100000:2.0:1789760777.035925:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707142832073 time 707155357333 state 0x2 ref 0 00000100:00100000:1.0:1789760777.042320:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.042635:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774720 00000100:00100000:3.0:1789760777.042653:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774720 00000100:00100000:3.0:1789760777.042717:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707142832073 time 707155357333 state 0x0 ref 1 00000100:00100000:3.0:1789760777.042735:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707155617749 time 707155357333 state 0x1 ref 1 00000100:00100000:3.0:1789760777.042763:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707155617749 time 707167370122 state 0x0 ref 1 00000100:00100000:3.0:1789760777.042776:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503774720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.042824:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76386304, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246e0 00010000:00010000:3.0:1789760777.042834:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.042856:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.042861:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.042867:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.042880:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.047969:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985956 is committed 00002000:00010000:3.0:1789760777.048136:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76386304 -> 76390400 00002000:00010000:3.0:1789760777.048141:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149216 -> 149224 00010000:00010000:3.0:1789760777.048154:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.048167:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.048176:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.048182:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.048187:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.048195:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.048214:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.048375:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f05f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76386304->76390399] (req 76386304->76390399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.048576:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503774720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5797us (6260us total) trans 4294985956 rc 0/0 00000100:00100000:3.0:1789760777.048587:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18660 00000100:00100000:3.0:1789760777.048597:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707155617749 time 707167370122 state 0x2 ref 0 00000100:00100000:1.0:1789760777.055202:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.055363:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774848 00000100:00100000:0.0:1789760777.055374:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774848 00000100:00100000:0.0:1789760777.055547:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707155617749 time 707167370122 state 0x0 ref 1 00000100:00100000:0.0:1789760777.055567:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707167630538 time 707167370122 state 0x1 ref 1 00000100:00100000:0.0:1789760777.055654:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707167630538 time 707180260326 state 0x0 ref 1 00000100:00100000:0.0:1789760777.055667:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.055714:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76390400, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246e8 00010000:00010000:0.0:1789760777.055723:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.055738:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.055743:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.055749:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.055763:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.059836:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985957 is committed 00002000:00010000:0.0:1789760777.059982:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76390400 -> 76394496 00002000:00010000:0.0:1789760777.059987:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149224 -> 149232 00010000:00010000:0.0:1789760777.060000:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.060013:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.060023:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.060037:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.060041:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.060051:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.060071:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.060079:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f05ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76390400->76394495] (req 76390400->76394495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.060175:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503774848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4505us (4978us total) trans 4294985957 rc 0/0 00000100:00100000:0.0:1789760777.060186:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18661 00000100:00100000:0.0:1789760777.060195:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707167630538 time 707180260326 state 0x2 ref 0 00000100:00100000:1.0:1789760777.068453:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.068760:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503774976 00000100:00100000:2.0:1789760777.068771:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503774976 00000100:00100000:2.0:1789760777.068949:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707167630538 time 707180260326 state 0x0 ref 1 00000100:00100000:2.0:1789760777.068966:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707180520742 time 707180260326 state 0x1 ref 1 00000100:00100000:2.0:1789760777.068992:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707180520742 time 707193598898 state 0x0 ref 1 00000100:00100000:2.0:1789760777.069006:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503774976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.069053:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76394496, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246f0 00010000:00010000:2.0:1789760777.069063:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069080:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.069084:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.069090:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069104:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.075858:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985958 is committed 00002000:00010000:2.0:1789760777.076040:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76394496 -> 76398592 00002000:00010000:2.0:1789760777.076045:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149232 -> 149240 00010000:00010000:2.0:1789760777.076058:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.076078:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.076087:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.076094:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.076099:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.076109:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.076129:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.076137:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0606 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76394496->76398591] (req 76394496->76398591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.076229:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503774976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7222us (7782us total) trans 4294985958 rc 0/0 00000100:00100000:2.0:1789760777.076240:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18662 00000100:00100000:2.0:1789760777.076249:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707180520742 time 707193598898 state 0x2 ref 0 00000100:00100000:1.0:1789760777.092595:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.092760:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503775104 00000100:00100000:2.0:1789760777.092772:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503775104 00000100:00100000:2.0:1789760777.092827:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707180520742 time 707193598898 state 0x0 ref 1 00000100:00100000:2.0:1789760777.092842:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707193859314 time 707193598898 state 0x1 ref 1 00000100:00100000:2.0:1789760777.092912:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707193859314 time 707217518364 state 0x0 ref 1 00000100:00100000:2.0:1789760777.092925:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503775104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.092971:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76398592, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x246f8 00010000:00010000:2.0:1789760777.092981:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.092997:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.093002:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.093008:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.093022:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.096857:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985959 is committed 00002000:00010000:2.0:1789760777.099666:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76398592 -> 76402688 00002000:00010000:2.0:1789760777.099677:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149240 -> 149248 00010000:00010000:2.0:1789760777.099692:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.099706:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.099721:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.099729:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.099734:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.099744:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.099764:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.099771:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f060d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76398592->76402687] (req 76398592->76402687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.099875:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503775104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6947us (7284us total) trans 4294985959 rc 0/0 00000100:00100000:2.0:1789760777.099885:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18663 00000100:00100000:2.0:1789760777.099895:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707193859314 time 707217518364 state 0x2 ref 0 00000100:00100000:1.0:1789760777.113246:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.113339:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503775232 00000100:00100000:3.0:1789760777.113349:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503775232 00000100:00100000:3.0:1789760777.113439:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707193859314 time 707217518364 state 0x0 ref 1 00000100:00100000:3.0:1789760777.113455:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707217778780 time 707217518364 state 0x1 ref 1 00000100:00100000:3.0:1789760777.113486:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707217778780 time 707238093164 state 0x0 ref 1 00000100:00100000:3.0:1789760777.113498:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503775232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.113538:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76402688, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24700 00010000:00010000:3.0:1789760777.113547:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.113562:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.113566:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.113572:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.113585:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.126699:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985960 is committed 00002000:00010000:3.0:1789760777.126863:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76402688 -> 76406784 00002000:00010000:3.0:1789760777.126869:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149248 -> 149256 00010000:00010000:3.0:1789760777.126882:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.126895:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.126909:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.126917:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.126922:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.126930:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.126950:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.126958:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0614 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76402688->76406783] (req 76402688->76406783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.127047:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503775232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13546us (13806us total) trans 4294985960 rc 0/0 00000100:00100000:3.0:1789760777.127057:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18664 00000100:00100000:3.0:1789760777.127080:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707217778780 time 707238093164 state 0x2 ref 0 00000100:00100000:1.0:1789760777.130824:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.130918:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503775360 00000100:00100000:0.0:1789760777.130929:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503775360 00000100:00100000:0.0:1789760777.130984:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707217778780 time 707238093164 state 0x0 ref 1 00000100:00100000:0.0:1789760777.130999:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707238353580 time 707238093164 state 0x1 ref 1 00000100:00100000:0.0:1789760777.131032:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707238353580 time 707255638347 state 0x0 ref 1 00000100:00100000:0.0:1789760777.131048:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503775360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.131105:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76406784, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24708 00010000:00010000:0.0:1789760777.131114:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.131130:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.131135:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.131140:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.131154:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.136804:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985961 is committed 00002000:00010000:0.0:1789760777.136923:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76406784 -> 76410880 00002000:00010000:0.0:1789760777.136928:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149256 -> 149264 00010000:00010000:0.0:1789760777.136941:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.136953:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.136962:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.136968:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.136973:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.136982:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.137000:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.137008:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f061b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76406784->76410879] (req 76406784->76410879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.137105:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503775360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6054us (6285us total) trans 4294985961 rc 0/0 00000100:00100000:0.0:1789760777.137115:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18665 00000100:00100000:0.0:1789760777.137125:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707238353580 time 707255638347 state 0x2 ref 0 00000100:00100000:1.0:1789760777.141598:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.141786:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503775488 00000100:00100000:2.0:1789760777.141796:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503775488 00000100:00100000:2.0:1789760777.141848:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707238353580 time 707255638347 state 0x0 ref 1 00000100:00100000:2.0:1789760777.141865:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707255898763 time 707255638347 state 0x1 ref 1 00000100:00100000:2.0:1789760777.141897:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707255898763 time 707266503158 state 0x0 ref 1 00000100:00100000:2.0:1789760777.141909:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503775488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.141958:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76410880, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24710 00010000:00010000:2.0:1789760777.141967:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.141983:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.141987:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.141994:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.142008:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.146751:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985962 is committed 00002000:00010000:2.0:1789760777.146870:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76410880 -> 76414976 00002000:00010000:2.0:1789760777.146875:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149264 -> 149272 00010000:00010000:2.0:1789760777.146888:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.146901:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.146910:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.146922:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.146926:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.146935:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.146953:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.146961:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0622 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76410880->76414975] (req 76410880->76414975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.147056:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503775488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5144us (5463us total) trans 4294985962 rc 0/0 00000100:00100000:2.0:1789760777.147066:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18666 00000100:00100000:2.0:1789760777.147087:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707255898763 time 707266503158 state 0x2 ref 0 00000100:00100000:1.0:1789760777.150771:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.150857:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503775616 00000100:00100000:2.0:1789760777.150866:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503775616 00000100:00100000:2.0:1789760777.150924:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707255898763 time 707266503158 state 0x0 ref 1 00000100:00100000:2.0:1789760777.150950:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707266763574 time 707266503158 state 0x1 ref 1 00000100:00100000:2.0:1789760777.150980:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707266763574 time 707275586888 state 0x0 ref 1 00000100:00100000:2.0:1789760777.150995:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503775616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.151040:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76414976, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24718 00010000:00010000:2.0:1789760777.151050:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.151066:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.151082:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.151088:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.151103:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.157074:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985963 is committed 00002000:00010000:1.0:1789760777.157260:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76414976 -> 76419072 00002000:00010000:1.0:1789760777.157266:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149272 -> 149280 00010000:00010000:1.0:1789760777.157279:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.157292:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.157300:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.157307:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.157312:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.157320:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.157457:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.157466:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0629 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76414976->76419071] (req 76414976->76419071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.157592:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503775616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6594us (6828us total) trans 4294985963 rc 0/0 00000100:00100000:1.0:1789760777.157613:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18667 00000100:00100000:1.0:1789760777.157623:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707266763574 time 707275586888 state 0x2 ref 0 00000100:00100000:1.0:1789760777.166183:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.167253:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503775744 00000100:00100000:3.0:1789760777.167268:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503775744 00000100:00100000:3.0:1789760777.167348:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707266763574 time 707275586888 state 0x0 ref 1 00000100:00100000:3.0:1789760777.167365:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707275847304 time 707275586888 state 0x1 ref 1 00000100:00100000:3.0:1789760777.167648:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707275847304 time 707292243195 state 0x0 ref 1 00000100:00100000:3.0:1789760777.167665:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503775744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.167709:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76419072, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24720 00010000:00010000:3.0:1789760777.167720:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.167734:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.167739:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.167746:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.167759:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.171730:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985964 is committed 00002000:00010000:0.0:1789760777.171915:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76419072 -> 76423168 00002000:00010000:0.0:1789760777.171921:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149280 -> 149288 00010000:00010000:0.0:1789760777.171935:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.171950:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.171959:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.171972:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.171977:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.171987:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.172007:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.172015:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0630 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76419072->76423167] (req 76419072->76423167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.172161:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503775744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4494us (5980us total) trans 4294985964 rc 0/0 00000100:00100000:0.0:1789760777.172172:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18668 00000100:00100000:0.0:1789760777.172183:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707275847304 time 707292243195 state 0x2 ref 0 00000100:00100000:2.0:1789760777.174882:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.175006:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503775872 00000100:00100000:0.0:1789760777.175016:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503775872 00000100:00100000:0.0:1789760777.175074:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707275847304 time 707292243195 state 0x0 ref 1 00000100:00100000:0.0:1789760777.175089:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707292503611 time 707292243195 state 0x1 ref 1 00000100:00100000:0.0:1789760777.175110:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707292503611 time 707299717310 state 0x0 ref 1 00000100:00100000:0.0:1789760777.175123:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503775872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.175169:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76423168, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24728 00010000:00010000:0.0:1789760777.175178:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.175194:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.175199:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.175205:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.175218:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.178883:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985965 is committed 00002000:00010000:0.0:1789760777.180745:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76423168 -> 76427264 00002000:00010000:0.0:1789760777.180753:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149288 -> 149296 00010000:00010000:0.0:1789760777.180767:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.180781:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.180791:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.180798:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.180804:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.180813:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.180833:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.180841:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0637 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76423168->76427263] (req 76423168->76427263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.181214:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503775872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6086us (6335us total) trans 4294985965 rc 0/0 00000100:00100000:0.0:1789760777.181236:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18669 00000100:00100000:0.0:1789760777.181246:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707292503611 time 707299717310 state 0x2 ref 0 00000100:00100000:0.0:1789760777.185505:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.185610:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776000 00000100:00100000:2.0:1789760777.185621:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776000 00000100:00100000:2.0:1789760777.185688:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707292503611 time 707299717310 state 0x0 ref 1 00000100:00100000:2.0:1789760777.185707:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707299977726 time 707299717310 state 0x1 ref 1 00000100:00100000:2.0:1789760777.185733:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707299977726 time 707310340437 state 0x0 ref 1 00000100:00100000:2.0:1789760777.185746:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503776000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.185877:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76427264, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24730 00010000:00010000:2.0:1789760777.185887:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185903:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.185907:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.185913:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185925:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.191548:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985966 is committed 00002000:00010000:2.0:1789760777.191868:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76427264 -> 76431360 00002000:00010000:2.0:1789760777.191874:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149296 -> 149304 00010000:00010000:2.0:1789760777.191887:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.191900:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.191909:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.191917:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.191921:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.191930:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.191951:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.191960:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f063e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76427264->76431359] (req 76427264->76431359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.192094:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503776000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6344us (6591us total) trans 4294985966 rc 0/0 00000100:00100000:2.0:1789760777.192106:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18670 00000100:00100000:2.0:1789760777.192117:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707299977726 time 707310340437 state 0x2 ref 0 00000100:00100000:1.0:1789760777.196924:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.197554:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776128 00000100:00100000:2.0:1789760777.197565:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776128 00000100:00100000:2.0:1789760777.197637:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707299977726 time 707310340437 state 0x0 ref 1 00000100:00100000:2.0:1789760777.197651:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707310600853 time 707310340437 state 0x1 ref 1 00000100:00100000:2.0:1789760777.197672:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707310600853 time 707322279119 state 0x0 ref 1 00000100:00100000:2.0:1789760777.197683:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.197726:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76431360, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24738 00010000:00010000:2.0:1789760777.197734:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.197749:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.197755:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.197761:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.197773:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.204928:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985967 is committed 00002000:00010000:2.0:1789760777.205093:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76431360 -> 76435456 00002000:00010000:2.0:1789760777.205098:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149304 -> 149312 00010000:00010000:2.0:1789760777.205111:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.205124:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.205132:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.205139:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.205144:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.205153:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.205171:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.205185:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0645 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76431360->76435455] (req 76431360->76435455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.205288:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7602us (8369us total) trans 4294985967 rc 0/0 00000100:00100000:2.0:1789760777.205297:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18671 00000100:00100000:2.0:1789760777.205307:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707310600853 time 707322279119 state 0x2 ref 0 00000100:00100000:1.0:1789760777.214431:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.214640:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776256 00000100:00100000:0.0:1789760777.214660:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776256 00000100:00100000:0.0:1789760777.214718:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707310600853 time 707322279119 state 0x0 ref 1 00000100:00100000:0.0:1789760777.214734:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707322539535 time 707322279119 state 0x1 ref 1 00000100:00100000:0.0:1789760777.214769:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707322539535 time 707339375292 state 0x0 ref 1 00000100:00100000:0.0:1789760777.214783:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.214833:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76435456, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24740 00010000:00010000:0.0:1789760777.214843:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.214860:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.214864:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.214871:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.214885:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.218789:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985968 is committed 00002000:00010000:0.0:1789760777.218908:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76435456 -> 76439552 00002000:00010000:0.0:1789760777.218914:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149312 -> 149320 00010000:00010000:0.0:1789760777.218927:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218941:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218950:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218956:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.218961:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218975:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.218995:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.219003:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f064c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76435456->76439551] (req 76435456->76439551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.219140:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4356us (4758us total) trans 4294985968 rc 0/0 00000100:00100000:0.0:1789760777.219151:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18672 00000100:00100000:0.0:1789760777.219160:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707322539535 time 707339375292 state 0x2 ref 0 00000100:00100000:0.0:1789760777.225728:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.225921:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776384 00000100:00100000:2.0:1789760777.225932:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776384 00000100:00100000:2.0:1789760777.225990:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707322539535 time 707339375292 state 0x0 ref 1 00000100:00100000:2.0:1789760777.226005:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707339635708 time 707339375292 state 0x1 ref 1 00000100:00100000:2.0:1789760777.226038:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707339635708 time 707350644404 state 0x0 ref 1 00000100:00100000:2.0:1789760777.226050:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503776384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.226100:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76439552, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24748 00010000:00010000:2.0:1789760777.226110:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.226126:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.226131:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.226136:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.226155:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.231498:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985969 is committed 00002000:00010000:0.0:1789760777.232065:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76439552 -> 76443648 00002000:00010000:0.0:1789760777.232069:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149320 -> 149328 00010000:00010000:0.0:1789760777.232080:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.232089:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.232095:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.232104:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.232108:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.232114:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.232129:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.232134:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0653 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76439552->76443647] (req 76439552->76443647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.232216:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503776384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6164us (6494us total) trans 4294985969 rc 0/0 00000100:00100000:0.0:1789760777.232223:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18673 00000100:00100000:0.0:1789760777.232233:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707339635708 time 707350644404 state 0x2 ref 0 00000100:00100000:1.0:1789760777.237256:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.237432:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776512 00000100:00100000:2.0:1789760777.237442:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776512 00000100:00100000:2.0:1789760777.237499:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707339635708 time 707350644404 state 0x0 ref 1 00000100:00100000:2.0:1789760777.237514:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707350904820 time 707350644404 state 0x1 ref 1 00000100:00100000:2.0:1789760777.237880:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707350904820 time 707362484149 state 0x0 ref 1 00000100:00100000:2.0:1789760777.237896:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.237946:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76443648, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24750 00010000:00010000:2.0:1789760777.237955:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237970:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.237974:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.237980:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237993:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.242177:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985970 is committed 00002000:00010000:2.0:1789760777.242492:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76443648 -> 76447744 00002000:00010000:2.0:1789760777.242499:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149328 -> 149336 00010000:00010000:2.0:1789760777.242512:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.242525:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.242533:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.242540:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.242545:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.242554:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.242574:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.242582:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f065a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76443648->76447743] (req 76443648->76447743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.242827:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4929us (5576us total) trans 4294985970 rc 0/0 00000100:00100000:2.0:1789760777.242839:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18674 00000100:00100000:2.0:1789760777.242849:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707350904820 time 707362484149 state 0x2 ref 0 00000100:00100000:1.0:1789760777.245938:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.245999:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776640 00000100:00100000:3.0:1789760777.246009:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776640 00000100:00100000:3.0:1789760777.246063:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707350904820 time 707362484149 state 0x0 ref 1 00000100:00100000:3.0:1789760777.246078:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707362744565 time 707362484149 state 0x1 ref 1 00000100:00100000:3.0:1789760777.246112:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707362744565 time 707370719190 state 0x0 ref 1 00000100:00100000:3.0:1789760777.246125:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503776640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.246169:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76447744, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24758 00010000:00010000:3.0:1789760777.246178:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.246194:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.246198:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.246204:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.246218:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.253131:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985971 is committed 00002000:00010000:3.0:1789760777.253385:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76447744 -> 76451840 00002000:00010000:3.0:1789760777.253407:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149336 -> 149344 00010000:00010000:3.0:1789760777.253421:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.253432:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.253440:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.253453:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.253458:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.253468:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.253486:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.253495:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0661 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76447744->76451839] (req 76447744->76451839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.253598:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503776640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7469us (7664us total) trans 4294985971 rc 0/0 00000100:00100000:3.0:1789760777.253619:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18675 00000100:00100000:3.0:1789760777.253628:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707362744565 time 707370719190 state 0x2 ref 0 00000100:00100000:0.0:1789760777.258690:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.260947:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776768 00000100:00100000:1.0:1789760777.260962:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776768 00000100:00100000:1.0:1789760777.261031:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707362744565 time 707370719190 state 0x0 ref 1 00000100:00100000:1.0:1789760777.261048:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707370979606 time 707370719190 state 0x1 ref 1 00000100:00100000:1.0:1789760777.261082:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707370979606 time 707385689046 state 0x0 ref 1 00000100:00100000:1.0:1789760777.261095:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.261142:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76451840, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24760 00010000:00010000:1.0:1789760777.261150:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.261165:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.261169:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.261176:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.261189:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.269243:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985972 is committed 00002000:00010000:1.0:1789760777.274804:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76451840 -> 76455936 00002000:00010000:1.0:1789760777.274816:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149344 -> 149352 00010000:00010000:1.0:1789760777.274830:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.274844:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.274853:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.274861:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.274865:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.274874:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.274894:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.274908:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0668 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76451840->76455935] (req 76451840->76455935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.275169:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503776768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14070us (16485us total) trans 4294985972 rc 0/0 00000100:00100000:1.0:1789760777.275181:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18676 00000100:00100000:1.0:1789760777.275192:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707370979606 time 707385689046 state 0x2 ref 0 00000100:00100000:1.0:1789760777.281551:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.281649:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503776896 00000100:00100000:0.0:1789760777.281659:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503776896 00000100:00100000:0.0:1789760777.281713:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707370979606 time 707385689046 state 0x0 ref 1 00000100:00100000:0.0:1789760777.281728:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707385949462 time 707385689046 state 0x1 ref 1 00000100:00100000:0.0:1789760777.281760:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707385949462 time 707406366667 state 0x0 ref 1 00000100:00100000:0.0:1789760777.281772:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503776896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.281819:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76455936, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24768 00010000:00010000:0.0:1789760777.281829:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.281845:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.281850:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.281857:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.281872:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.291293:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985973 is committed 00002000:00010000:0.0:1789760777.291545:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76455936 -> 76460032 00002000:00010000:0.0:1789760777.291550:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149352 -> 149360 00010000:00010000:0.0:1789760777.291563:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.291576:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.291585:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.291592:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.291597:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.291617:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.291635:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.291643:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f066f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76455936->76460031] (req 76455936->76460031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.291736:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503776896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9961us (10190us total) trans 4294985973 rc 0/0 00000100:00100000:0.0:1789760777.291747:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18677 00000100:00100000:0.0:1789760777.291756:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707385949462 time 707406366667 state 0x2 ref 0 00000100:00100000:2.0:1789760777.301520:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.301634:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777024 00000100:00100000:1.0:1789760777.301643:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777024 00000100:00100000:1.0:1789760777.301697:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707385949462 time 707406366667 state 0x0 ref 1 00000100:00100000:1.0:1789760777.301712:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707406627083 time 707406366667 state 0x1 ref 1 00000100:00100000:1.0:1789760777.301735:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707406627083 time 707426341148 state 0x0 ref 1 00000100:00100000:1.0:1789760777.301747:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503777024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.301792:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76460032, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24770 00010000:00010000:1.0:1789760777.301801:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.301815:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.301820:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.301827:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.301840:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.308617:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985974 is committed 00002000:00010000:1.0:1789760777.308898:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76460032 -> 76464128 00002000:00010000:1.0:1789760777.308905:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149360 -> 149368 00010000:00010000:1.0:1789760777.308923:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.308939:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.308948:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.308956:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.308971:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.308983:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.309007:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.309016:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0676 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76460032->76464127] (req 76460032->76464127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.309183:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503777024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7431us (7666us total) trans 4294985974 rc 0/0 00000100:00100000:1.0:1789760777.309197:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18678 00000100:00100000:1.0:1789760777.309209:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707406627083 time 707426341148 state 0x2 ref 0 00000100:00100000:3.0:1789760777.322726:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.322829:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777152 00000100:00100000:2.0:1789760777.322847:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777152 00000100:00100000:2.0:1789760777.322928:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707406627083 time 707426341148 state 0x0 ref 1 00000100:00100000:2.0:1789760777.322957:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707426601564 time 707426341148 state 0x1 ref 1 00000100:00100000:2.0:1789760777.322997:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707426601564 time 707447603557 state 0x0 ref 1 00000100:00100000:2.0:1789760777.323013:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.323075:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76464128, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24778 00010000:00010000:2.0:1789760777.323086:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.323108:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.323114:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.323122:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.323139:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.335847:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985975 is committed 00002000:00010000:3.0:1789760777.336839:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76464128 -> 76468224 00002000:00010000:3.0:1789760777.336847:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149368 -> 149376 00010000:00010000:3.0:1789760777.336860:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336874:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336883:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336890:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.336894:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336903:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336922:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336929:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f067d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76464128->76468223] (req 76464128->76468223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.337030:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14016us (14307us total) trans 4294985975 rc 0/0 00000100:00100000:3.0:1789760777.337039:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18679 00000100:00100000:3.0:1789760777.337049:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707426601564 time 707447603557 state 0x2 ref 0 00000100:00100000:3.0:1789760777.351457:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.351645:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777280 00000100:00100000:0.0:1789760777.351654:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777280 00000100:00100000:0.0:1789760777.351712:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707426601564 time 707447603557 state 0x0 ref 1 00000100:00100000:0.0:1789760777.351727:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707447863973 time 707447603557 state 0x1 ref 1 00000100:00100000:0.0:1789760777.351757:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707447863973 time 707476363771 state 0x0 ref 1 00000100:00100000:0.0:1789760777.351769:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503777280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.351815:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76468224, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24780 00010000:00010000:0.0:1789760777.351825:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.351839:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.351845:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.351851:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.351864:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.359187:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985976 is committed 00002000:00010000:0.0:1789760777.359312:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76468224 -> 76472320 00002000:00010000:0.0:1789760777.359318:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149376 -> 149384 00010000:00010000:0.0:1789760777.359331:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.359344:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.359353:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.359360:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.359442:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.359453:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.359474:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.359482:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0684 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76468224->76472319] (req 76468224->76472319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.359640:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503777280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7868us (8256us total) trans 4294985976 rc 0/0 00000100:00100000:0.0:1789760777.359651:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18680 00000100:00100000:0.0:1789760777.359661:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707447863973 time 707476363771 state 0x2 ref 0 00000100:00100000:2.0:1789760777.364901:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.369535:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777408 00000100:00100000:3.0:1789760777.369550:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777408 00000100:00100000:3.0:1789760777.369783:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707447863973 time 707476363771 state 0x0 ref 1 00000100:00100000:3.0:1789760777.369801:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707476624187 time 707476363771 state 0x1 ref 1 00000100:00100000:3.0:1789760777.369889:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707476624187 time 707494495452 state 0x0 ref 1 00000100:00100000:3.0:1789760777.369903:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.369950:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76472320, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24788 00010000:00010000:3.0:1789760777.369959:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.369975:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.369979:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.369986:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.370000:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.377744:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985977 is committed 00002000:00010000:0.0:1789760777.382999:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76472320 -> 76476416 00002000:00010000:0.0:1789760777.383011:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149384 -> 149392 00010000:00010000:0.0:1789760777.383028:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.383044:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.383054:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.383063:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.383068:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.383084:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.383111:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.383120:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f068b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76472320->76476415] (req 76472320->76476415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.383308:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13401us (18409us total) trans 4294985977 rc 0/0 00000100:00100000:0.0:1789760777.383322:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18681 00000100:00100000:0.0:1789760777.383335:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707476624187 time 707494495452 state 0x2 ref 0 00000100:00100000:3.0:1789760777.388523:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.388762:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777536 00000100:00100000:1.0:1789760777.388774:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777536 00000100:00100000:1.0:1789760777.388849:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707476624187 time 707494495452 state 0x0 ref 1 00000100:00100000:1.0:1789760777.388867:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707494755868 time 707494495452 state 0x1 ref 1 00000100:00100000:1.0:1789760777.388903:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707494755868 time 707513509082 state 0x0 ref 1 00000100:00100000:1.0:1789760777.388918:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.388991:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76476416, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24790 00010000:00010000:1.0:1789760777.389003:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.389022:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.389028:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.389035:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.389052:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.401719:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985978 is committed 00002000:00010000:1.0:1789760777.401947:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76476416 -> 76480512 00002000:00010000:1.0:1789760777.401953:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149392 -> 149400 00010000:00010000:1.0:1789760777.401971:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.401987:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.401998:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.402006:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.402012:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.402022:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.402050:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.402058:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0692 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76476416->76480511] (req 76476416->76480511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.402187:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13265us (13673us total) trans 4294985978 rc 0/0 00000100:00100000:1.0:1789760777.402201:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18682 00000100:00100000:1.0:1789760777.402215:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707494755868 time 707513509082 state 0x2 ref 0 00000100:00100000:3.0:1789760777.410151:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.410276:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777664 00000100:00100000:2.0:1789760777.410291:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777664 00000100:00100000:2.0:1789760777.410378:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707494755868 time 707513509082 state 0x0 ref 1 00000100:00100000:2.0:1789760777.410448:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707513769498 time 707513509082 state 0x1 ref 1 00000100:00100000:2.0:1789760777.410483:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707513769498 time 707535089538 state 0x0 ref 1 00000100:00100000:2.0:1789760777.410500:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.410574:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76480512, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24798 00010000:00010000:2.0:1789760777.410587:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.410616:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.410622:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.410630:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.410654:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.423212:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985979 is committed 00002000:00010000:1.0:1789760777.423364:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76480512 -> 76484608 00002000:00010000:1.0:1789760777.423370:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149400 -> 149408 00010000:00010000:1.0:1789760777.423408:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.423425:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.423435:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.423444:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.423451:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.423462:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.423488:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.423496:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0699 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76480512->76484607] (req 76480512->76484607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.423705:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13202us (13562us total) trans 4294985979 rc 0/0 00000100:00100000:1.0:1789760777.423720:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18683 00000100:00100000:1.0:1789760777.423733:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707513769498 time 707535089538 state 0x2 ref 0 00000100:00100000:2.0:1789760777.436793:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.436966:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777792 00000100:00100000:0.0:1789760777.436978:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777792 00000100:00100000:0.0:1789760777.437058:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707513769498 time 707535089538 state 0x0 ref 1 00000100:00100000:0.0:1789760777.437076:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707535349954 time 707535089538 state 0x1 ref 1 00000100:00100000:0.0:1789760777.437114:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707535349954 time 707561720280 state 0x0 ref 1 00000100:00100000:0.0:1789760777.437129:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.437194:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76484608, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247a0 00010000:00010000:0.0:1789760777.437207:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.437226:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.437232:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.437240:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.437258:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.450058:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985980 is committed 00002000:00010000:0.0:1789760777.450216:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76484608 -> 76488704 00002000:00010000:0.0:1789760777.450223:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149408 -> 149416 00010000:00010000:0.0:1789760777.450240:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.450256:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.450267:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.450275:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.450281:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.450293:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.450332:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.450341:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76484608->76488703] (req 76484608->76488703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.450535:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503777792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13401us (13748us total) trans 4294985980 rc 0/0 00000100:00100000:0.0:1789760777.450548:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18684 00000100:00100000:0.0:1789760777.450560:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707535349954 time 707561720280 state 0x2 ref 0 00000100:00100000:3.0:1789760777.455563:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.457307:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503777920 00000100:00100000:0.0:1789760777.457336:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503777920 00000100:00100000:0.0:1789760777.457443:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707535349954 time 707561720280 state 0x0 ref 1 00000100:00100000:0.0:1789760777.457464:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707561980696 time 707561720280 state 0x1 ref 1 00000100:00100000:0.0:1789760777.457549:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707561980696 time 707582154257 state 0x0 ref 1 00000100:00100000:0.0:1789760777.457565:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503777920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.457656:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76488704, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247a8 00010000:00010000:0.0:1789760777.457667:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.457687:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.457693:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.457702:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.457718:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.466459:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985981 is committed 00002000:00010000:0.0:1789760777.466687:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76488704 -> 76492800 00002000:00010000:0.0:1789760777.466693:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149416 -> 149424 00010000:00010000:0.0:1789760777.466712:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.466728:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.466737:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.466746:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.466751:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.466762:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.466789:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.466803:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76488704->76492799] (req 76488704->76492799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.467376:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503777920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9806us (11820us total) trans 4294985981 rc 0/0 00000100:00100000:0.0:1789760777.467450:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18685 00000100:00100000:0.0:1789760777.467464:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707561980696 time 707582154257 state 0x2 ref 0 00000100:00100000:3.0:1789760777.475212:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.475674:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778048 00000100:00100000:1.0:1789760777.475687:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778048 00000100:00100000:1.0:1789760777.475769:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707561980696 time 707582154257 state 0x0 ref 1 00000100:00100000:1.0:1789760777.475789:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707582414673 time 707582154257 state 0x1 ref 1 00000100:00100000:1.0:1789760777.475829:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707582414673 time 707600435293 state 0x0 ref 1 00000100:00100000:1.0:1789760777.475843:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.475909:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76492800, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247b0 00010000:00010000:1.0:1789760777.475921:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.475944:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.475949:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.475957:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.475974:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.484349:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985982 is committed 00002000:00010000:1.0:1789760777.485280:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76492800 -> 76496896 00002000:00010000:1.0:1789760777.485290:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149424 -> 149432 00010000:00010000:1.0:1789760777.485309:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.485480:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.485493:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.485502:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.485507:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.485519:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.485545:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.485555:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f06ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76492800->76496895] (req 76492800->76496895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.485756:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9907us (10554us total) trans 4294985982 rc 0/0 00000100:00100000:1.0:1789760777.485770:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18686 00000100:00100000:1.0:1789760777.485784:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707582414673 time 707600435293 state 0x2 ref 0 00000100:00100000:1.0:1789760777.495989:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.496122:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778176 00000100:00100000:3.0:1789760777.496137:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778176 00000100:00100000:3.0:1789760777.496211:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707582414673 time 707600435293 state 0x0 ref 1 00000100:00100000:3.0:1789760777.496231:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707600695709 time 707600435293 state 0x1 ref 1 00000100:00100000:3.0:1789760777.496272:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707600695709 time 707620877983 state 0x0 ref 1 00000100:00100000:3.0:1789760777.496286:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503778176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.496352:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76496896, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247b8 00010000:00010000:3.0:1789760777.496363:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.496384:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.496677:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.496687:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.496707:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.505103:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985983 is committed 00002000:00010000:3.0:1789760777.505263:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76496896 -> 76500992 00002000:00010000:3.0:1789760777.505270:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149432 -> 149440 00010000:00010000:3.0:1789760777.505288:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.505304:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.505316:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.505337:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.505344:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.505355:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.505381:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.505422:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f06b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76496896->76500991] (req 76496896->76500991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.505571:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503778176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9279us (9594us total) trans 4294985983 rc 0/0 00000100:00100000:3.0:1789760777.505584:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18687 00000100:00100000:3.0:1789760777.505597:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707600695709 time 707620877983 state 0x2 ref 0 00000100:00100000:3.0:1789760777.510782:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.511823:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778304 00000100:00100000:0.0:1789760777.511843:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778304 00000100:00100000:0.0:1789760777.511927:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707600695709 time 707620877983 state 0x0 ref 1 00000100:00100000:0.0:1789760777.511948:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707621138399 time 707620877983 state 0x1 ref 1 00000100:00100000:0.0:1789760777.512035:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707621138399 time 707636640777 state 0x0 ref 1 00000100:00100000:0.0:1789760777.512051:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.512115:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76500992, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247c0 00010000:00010000:0.0:1789760777.512127:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512148:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.512154:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.512162:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512180:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.520850:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985984 is committed 00002000:00010000:1.0:1789760777.521155:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76500992 -> 76505088 00002000:00010000:1.0:1789760777.521162:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149440 -> 149448 00010000:00010000:1.0:1789760777.521179:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.521194:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.521203:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.521210:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.521216:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.521227:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.521254:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.521262:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f06bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76500992->76505087] (req 76500992->76505087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.521695:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9638us (10920us total) trans 4294985984 rc 0/0 00000100:00100000:1.0:1789760777.521710:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18688 00000100:00100000:1.0:1789760777.521722:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707621138399 time 707636640777 state 0x2 ref 0 00000100:00100000:1.0:1789760777.526754:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.530584:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778432 00000100:00100000:0.0:1789760777.530621:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778432 00000100:00100000:0.0:1789760777.530979:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707621138399 time 707636640777 state 0x0 ref 1 00000100:00100000:0.0:1789760777.530998:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707636901193 time 707636640777 state 0x1 ref 1 00000100:00100000:0.0:1789760777.531088:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707636901193 time 707655693355 state 0x0 ref 1 00000100:00100000:0.0:1789760777.531106:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503778432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.531175:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76505088, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247c8 00010000:00010000:0.0:1789760777.531188:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.531208:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.531213:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.531220:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.531238:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.539041:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985985 is committed 00002000:00010000:3.0:1789760777.539234:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76505088 -> 76509184 00002000:00010000:3.0:1789760777.539242:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149448 -> 149456 00010000:00010000:3.0:1789760777.539262:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.539279:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.539289:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.539298:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.539303:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.539314:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.539451:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.539461:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f06c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76505088->76509183] (req 76505088->76509183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.539584:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503778432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8474us (12841us total) trans 4294985985 rc 0/0 00000100:00100000:3.0:1789760777.539596:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18689 00000100:00100000:3.0:1789760777.539618:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707636901193 time 707655693355 state 0x2 ref 0 00000100:00100000:3.0:1789760777.544909:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.545881:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778560 00000100:00100000:1.0:1789760777.545899:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778560 00000100:00100000:1.0:1789760777.545978:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707636901193 time 707655693355 state 0x0 ref 1 00000100:00100000:1.0:1789760777.546062:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707655953771 time 707655693355 state 0x1 ref 1 00000100:00100000:1.0:1789760777.546100:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707655953771 time 707670706912 state 0x0 ref 1 00000100:00100000:1.0:1789760777.546115:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.546178:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76509184, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247d0 00010000:00010000:1.0:1789760777.546189:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.546207:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.546212:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.546219:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.546235:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.555103:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985986 is committed 00002000:00010000:1.0:1789760777.555211:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76509184 -> 76513280 00002000:00010000:1.0:1789760777.555215:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149456 -> 149464 00010000:00010000:1.0:1789760777.555231:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.555243:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.555249:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.555254:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.555259:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.555266:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.555286:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.555290:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f06ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76509184->76513279] (req 76509184->76513279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.555464:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9344us (10563us total) trans 4294985986 rc 0/0 00000100:00100000:1.0:1789760777.555591:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18690 00000100:00100000:1.0:1789760777.555614:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707655953771 time 707670706912 state 0x2 ref 0 00000100:00100000:0.0:1789760777.558896:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.559004:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778688 00000100:00100000:3.0:1789760777.559018:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778688 00000100:00100000:3.0:1789760777.559097:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707655953771 time 707670706912 state 0x0 ref 1 00000100:00100000:3.0:1789760777.559116:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707670967328 time 707670706912 state 0x1 ref 1 00000100:00100000:3.0:1789760777.559153:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707670967328 time 707683759548 state 0x0 ref 1 00000100:00100000:3.0:1789760777.559165:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.559229:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76513280, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247d8 00010000:00010000:3.0:1789760777.559240:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.559265:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.559271:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.559279:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.559296:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.565121:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985987 is committed 00002000:00010000:3.0:1789760777.565269:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76513280 -> 76517376 00002000:00010000:3.0:1789760777.565275:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149464 -> 149472 00010000:00010000:3.0:1789760777.565293:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.565308:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.565318:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.565337:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.565342:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.565352:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.565376:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.565383:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f06d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76513280->76517375] (req 76513280->76517375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.565753:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6581us (6865us total) trans 4294985987 rc 0/0 00000100:00100000:3.0:1789760777.565766:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18691 00000100:00100000:3.0:1789760777.565778:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707670967328 time 707683759548 state 0x2 ref 0 00000100:00100000:3.0:1789760777.569648:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.569924:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778816 00000100:00100000:0.0:1789760777.569938:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778816 00000100:00100000:0.0:1789760777.570024:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707670967328 time 707683759548 state 0x0 ref 1 00000100:00100000:0.0:1789760777.570044:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707684019964 time 707683759548 state 0x1 ref 1 00000100:00100000:0.0:1789760777.570081:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707684019964 time 707694687400 state 0x0 ref 1 00000100:00100000:0.0:1789760777.570096:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.570160:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76517376, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247e0 00010000:00010000:0.0:1789760777.570171:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.570193:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.570199:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.570207:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.570224:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.579185:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985988 is committed 00002000:00010000:0.0:1789760777.579570:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76517376 -> 76521472 00002000:00010000:0.0:1789760777.579578:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149472 -> 149480 00010000:00010000:0.0:1789760777.579596:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579626:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579636:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579644:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.579650:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579660:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579687:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579696:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76517376->76521471] (req 76517376->76521471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.579829:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503778816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9729us (10197us total) trans 4294985988 rc 0/0 00000100:00100000:0.0:1789760777.579843:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18692 00000100:00100000:0.0:1789760777.579856:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707684019964 time 707694687400 state 0x2 ref 0 00000100:00100000:1.0:1789760777.590044:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.590217:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503778944 00000100:00100000:0.0:1789760777.590230:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503778944 00000100:00100000:0.0:1789760777.590378:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707684019964 time 707694687400 state 0x0 ref 1 00000100:00100000:0.0:1789760777.590417:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707694947816 time 707694687400 state 0x1 ref 1 00000100:00100000:0.0:1789760777.590457:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707694947816 time 707715062905 state 0x0 ref 1 00000100:00100000:0.0:1789760777.590473:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503778944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.590538:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76521472, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247e8 00010000:00010000:0.0:1789760777.590552:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.590573:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.590579:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.590586:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.591330:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.597364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985989 is committed 00002000:00010000:0.0:1789760777.599081:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76521472 -> 76525568 00002000:00010000:0.0:1789760777.599093:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149480 -> 149488 00010000:00010000:0.0:1789760777.599111:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.599127:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.599138:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.599147:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.599153:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.599163:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.599186:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.599196:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76521472->76525567] (req 76521472->76525567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.599332:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503778944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8856us (9296us total) trans 4294985989 rc 0/0 00000100:00100000:0.0:1789760777.599344:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18693 00000100:00100000:0.0:1789760777.599357:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707694947816 time 707715062905 state 0x2 ref 0 00000100:00100000:3.0:1789760777.606081:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.606160:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779072 00000100:00100000:1.0:1789760777.606175:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779072 00000100:00100000:1.0:1789760777.606268:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707694947816 time 707715062905 state 0x0 ref 1 00000100:00100000:1.0:1789760777.606289:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707715323321 time 707715062905 state 0x1 ref 1 00000100:00100000:1.0:1789760777.606326:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707715323321 time 707730932040 state 0x0 ref 1 00000100:00100000:1.0:1789760777.606341:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503779072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.606434:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76525568, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247f0 00010000:00010000:1.0:1789760777.606445:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.606472:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.606478:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.606486:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.606503:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.613374:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985990 is committed 00002000:00010000:1.0:1789760777.614547:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76525568 -> 76529664 00002000:00010000:1.0:1789760777.614557:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149488 -> 149496 00010000:00010000:1.0:1789760777.614575:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.614588:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.615438:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.615460:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.615466:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.615478:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.615506:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.615514:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f06e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76525568->76529663] (req 76525568->76529663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.615670:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503779072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9321us (9595us total) trans 4294985990 rc 0/0 00000100:00100000:1.0:1789760777.615683:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18694 00000100:00100000:1.0:1789760777.615697:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707715323321 time 707730932040 state 0x2 ref 0 00000100:00100000:3.0:1789760777.622265:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.622575:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779200 00000100:00100000:1.0:1789760777.622588:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779200 00000100:00100000:1.0:1789760777.623111:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707715323321 time 707730932040 state 0x0 ref 1 00000100:00100000:1.0:1789760777.623131:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707731192456 time 707730932040 state 0x1 ref 1 00000100:00100000:1.0:1789760777.623221:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707731192456 time 707747826219 state 0x0 ref 1 00000100:00100000:1.0:1789760777.623235:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503779200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.623324:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76529664, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x247f8 00010000:00010000:1.0:1789760777.623335:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.623356:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.623361:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.623369:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.623384:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.628031:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985991 is committed 00002000:00010000:1.0:1789760777.628174:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76529664 -> 76533760 00002000:00010000:1.0:1789760777.628180:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149496 -> 149504 00010000:00010000:1.0:1789760777.628199:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.628220:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.628231:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.628238:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.628244:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.628254:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.628280:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.628288:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f06ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76529664->76533759] (req 76529664->76533759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.628492:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503779200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5252us (6235us total) trans 4294985991 rc 0/0 00000100:00100000:1.0:1789760777.628504:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18695 00000100:00100000:1.0:1789760777.628528:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707731192456 time 707747826219 state 0x2 ref 0 00000100:00100000:3.0:1789760777.638105:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.638691:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779328 00000100:00100000:0.0:1789760777.638707:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779328 00000100:00100000:0.0:1789760777.638790:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707731192456 time 707747826219 state 0x0 ref 1 00000100:00100000:0.0:1789760777.638809:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707748086635 time 707747826219 state 0x1 ref 1 00000100:00100000:0.0:1789760777.638889:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707748086635 time 707763494031 state 0x0 ref 1 00000100:00100000:0.0:1789760777.638904:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503779328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.638973:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76533760, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24800 00010000:00010000:0.0:1789760777.638983:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.639005:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.639010:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.639018:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.639033:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.643378:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985992 is committed 00002000:00010000:0.0:1789760777.648013:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76533760 -> 76537856 00002000:00010000:0.0:1789760777.648027:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149504 -> 149512 00010000:00010000:0.0:1789760777.648046:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.648069:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.648079:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.648087:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.648092:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.648102:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.648127:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.648136:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f06f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76533760->76537855] (req 76533760->76537855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.648343:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503779328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9435us (10246us total) trans 4294985992 rc 0/0 00000100:00100000:0.0:1789760777.648358:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18696 00000100:00100000:0.0:1789760777.648371:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707748086635 time 707763494031 state 0x2 ref 0 00000100:00100000:3.0:1789760777.651940:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.652023:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779456 00000100:00100000:0.0:1789760777.652036:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779456 00000100:00100000:0.0:1789760777.652109:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707748086635 time 707763494031 state 0x0 ref 1 00000100:00100000:0.0:1789760777.652128:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707763754447 time 707763494031 state 0x1 ref 1 00000100:00100000:0.0:1789760777.652167:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707763754447 time 707776773108 state 0x0 ref 1 00000100:00100000:0.0:1789760777.652182:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503779456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.652250:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76537856, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24808 00010000:00010000:0.0:1789760777.652262:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.652283:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.652288:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.652296:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.652315:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.658743:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985993 is committed 00002000:00010000:0.0:1789760777.658868:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76537856 -> 76541952 00002000:00010000:0.0:1789760777.658873:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149512 -> 149520 00010000:00010000:0.0:1789760777.658890:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.658901:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.658913:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.658919:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.658923:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.658929:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.658948:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.658953:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f06fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76537856->76541951] (req 76537856->76541951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.659101:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503779456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6915us (7166us total) trans 4294985993 rc 0/0 00000100:00100000:0.0:1789760777.659112:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18697 00000100:00100000:0.0:1789760777.659126:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707763754447 time 707776773108 state 0x2 ref 0 00000100:00100000:3.0:1789760777.662979:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.663135:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779584 00000100:00100000:1.0:1789760777.663149:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779584 00000100:00100000:1.0:1789760777.663216:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707763754447 time 707776773108 state 0x0 ref 1 00000100:00100000:1.0:1789760777.663231:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707777033524 time 707776773108 state 0x1 ref 1 00000100:00100000:1.0:1789760777.663295:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707777033524 time 707787900804 state 0x0 ref 1 00000100:00100000:1.0:1789760777.663309:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503779584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.663386:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76541952, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24810 00010000:00010000:1.0:1789760777.663432:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.663452:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.663457:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.663465:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.663481:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.669195:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.669537:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779712 00000100:00100000:2.0:1789760777.669553:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779712 00000100:00100000:2.0:1789760777.669647:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 488 00000100:00100000:2.0:1789760777.669659:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503779712:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760777.669762:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503779712:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 104us (580us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760777.669770:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 488 00000100:00100000:2.0:1789760777.670447:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.670478:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.670551:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779840 00000100:00100000:1.0:1789760777.670558:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779840 00000100:00100000:1.0:1789760777.670736:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780096 00000100:00100000:1.0:1789760777.670740:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780096 00000100:00100000:1.0:1789760777.670772:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 961 00000100:00100000:1.0:1789760777.670780:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2393:x1876699503779840:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:1.0:1789760777.670847:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2393:x1876699503779840:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 67us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.670855:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 961 00000100:00100000:1.0:1789760777.670871:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 962 00000100:00100000:1.0:1789760777.670877:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503780096:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:1.0:1789760777.670904:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503780096:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 28us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.670910:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 962 00000100:00080000:2.0:1789760777.671016:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760777.672452:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985994 is committed 00000100:00100000:0.0:1789760777.672908:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.672952:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.673028:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503779968 00000100:00100000:1.0:1789760777.673044:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503779968 00000100:00100000:1.0:1789760777.673119:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780224 00000100:00100000:1.0:1789760777.673122:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780224 00000100:00100000:1.0:1789760777.673147:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 963 00000100:00100000:1.0:1789760777.673157:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503779968:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:1.0:1789760777.673250:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503779968:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 94us (351us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.673259:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 963 00000100:00100000:1.0:1789760777.673278:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 964 00000100:00100000:1.0:1789760777.673284:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503780224:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:1.0:1789760777.673309:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503780224:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 26us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.673316:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 964 00000100:00080000:0.0:1789760777.674423:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00002000:00010000:2.0:1789760777.674824:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76541952 -> 76546048 00002000:00010000:2.0:1789760777.674832:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149520 -> 149528 00010000:00010000:2.0:1789760777.674850:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.674864:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.674874:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.674888:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.674893:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.674904:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.674929:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.674938:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0702 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76541952->76546047] (req 76541952->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.675027:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503779584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11716us (12058us total) trans 4294985994 rc 0/0 00000100:00100000:2.0:1789760777.675038:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18698 00000100:00100000:2.0:1789760777.675050:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707777033524 time 707787900804 state 0x2 ref 0 00000100:00100000:0.0:1789760777.680000:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.680182:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780352 00000100:00100000:1.0:1789760777.680195:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780352 00000100:00100000:1.0:1789760777.680275:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707777033524 time 707787900804 state 0x0 ref 1 00000100:00100000:1.0:1789760777.680376:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707788161220 time 707787900804 state 0x1 ref 1 00000100:00100000:0.0:1789760777.680448:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707788161220 time 707805047438 state 0x0 ref 1 00000100:00100000:0.0:1789760777.680465:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.680531:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76546048, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24818 00010000:00010000:0.0:1789760777.680543:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.680564:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.680569:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.680575:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.680614:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.686065:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985995 is committed 00002000:00010000:0.0:1789760777.686217:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76546048 -> 76550144 00002000:00010000:0.0:1789760777.686224:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149528 -> 149536 00010000:00010000:0.0:1789760777.686239:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.686254:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.686265:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.686278:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.686283:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.686293:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.686315:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.686328:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0709 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76546048->76550143] (req 76546048->76550143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.686545:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6078us (6552us total) trans 4294985995 rc 0/0 00000100:00100000:0.0:1789760777.686557:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18699 00000100:00100000:0.0:1789760777.686570:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707788161220 time 707805047438 state 0x2 ref 0 00000100:00100000:0.0:1789760777.691459:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.691565:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780480 00000100:00100000:1.0:1789760777.691580:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780480 00000100:00100000:1.0:1789760777.691672:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707788161220 time 707805047438 state 0x0 ref 1 00000100:00100000:1.0:1789760777.691691:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707805307854 time 707805047438 state 0x1 ref 1 00000100:00100000:1.0:1789760777.691730:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707805307854 time 707816335863 state 0x0 ref 1 00000100:00100000:1.0:1789760777.691745:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.691823:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76550144, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24820 00010000:00010000:1.0:1789760777.691834:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.691853:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.691858:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.691865:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.691882:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.697143:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985996 is committed 00002000:00010000:1.0:1789760777.697250:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76550144 -> 76554240 00002000:00010000:1.0:1789760777.697257:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149536 -> 149544 00010000:00010000:1.0:1789760777.697276:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.697290:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.697299:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.697306:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.697312:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.697332:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.697358:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.697365:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0710 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76550144->76554239] (req 76550144->76554239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.698091:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6339us (6637us total) trans 4294985996 rc 0/0 00000100:00100000:1.0:1789760777.698113:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18700 00000100:00100000:1.0:1789760777.698127:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707805307854 time 707816335863 state 0x2 ref 0 00000100:00100000:1.0:1789760777.705543:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.705621:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780608 00000100:00100000:3.0:1789760777.705647:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780608 00000100:00100000:3.0:1789760777.705726:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707805307854 time 707816335863 state 0x0 ref 1 00000100:00100000:3.0:1789760777.705744:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707816596279 time 707816335863 state 0x1 ref 1 00000100:00100000:3.0:1789760777.705783:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707816596279 time 707830389806 state 0x0 ref 1 00000100:00100000:3.0:1789760777.705798:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503780608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.706098:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76554240, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24828 00010000:00010000:3.0:1789760777.706111:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.706131:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.706136:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.706143:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.706160:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.713369:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985997 is committed 00002000:00010000:3.0:1789760777.713781:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76554240 -> 76558336 00002000:00010000:3.0:1789760777.713788:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149544 -> 149552 00010000:00010000:3.0:1789760777.713814:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.713831:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.713842:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.713850:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760777.713855:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.713866:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.713892:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.713900:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0717 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76554240->76558335] (req 76554240->76558335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.714055:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503780608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8253us (8519us total) trans 4294985997 rc 0/0 00000100:00100000:3.0:1789760777.714069:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18701 00000100:00100000:3.0:1789760777.714081:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707816596279 time 707830389806 state 0x2 ref 0 00000100:00100000:0.0:1789760777.720090:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.721510:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780736 00000100:00100000:2.0:1789760777.721532:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780736 00000100:00100000:2.0:1789760777.721629:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707816596279 time 707830389806 state 0x0 ref 1 00000100:00100000:2.0:1789760777.721653:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707830650222 time 707830389806 state 0x1 ref 1 00000100:00100000:2.0:1789760777.721700:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707830650222 time 707846306727 state 0x0 ref 1 00000100:00100000:2.0:1789760777.721715:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.721783:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76558336, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24830 00010000:00010000:2.0:1789760777.721796:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.721817:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.721823:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.721832:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.721849:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.727899:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985998 is committed 00002000:00010000:2.0:1789760777.728338:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76558336 -> 76562432 00002000:00010000:2.0:1789760777.728346:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149552 -> 149560 00010000:00010000:2.0:1789760777.728364:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.728381:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.728437:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.728452:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.728458:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.728469:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.728497:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.728506:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f071e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76558336->76562431] (req 76558336->76562431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.728831:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7111us (8748us total) trans 4294985998 rc 0/0 00000100:00100000:2.0:1789760777.728846:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18702 00000100:00100000:2.0:1789760777.728860:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707830650222 time 707846306727 state 0x2 ref 0 00000100:00100000:0.0:1789760777.734956:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.740585:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780864 00000100:00100000:3.0:1789760777.740617:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780864 00000100:00100000:3.0:1789760777.740899:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707830650222 time 707846306727 state 0x0 ref 1 00000100:00100000:3.0:1789760777.740916:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707846567143 time 707846306727 state 0x1 ref 1 00000100:00100000:3.0:1789760777.740952:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707846567143 time 707865559394 state 0x0 ref 1 00000100:00100000:3.0:1789760777.740967:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503780864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.741038:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76562432, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24838 00010000:00010000:3.0:1789760777.741050:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.741070:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.741076:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.741084:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.741100:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.747170:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760777.747955:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294985999 is committed 00002000:00010000:2.0:1789760777.748189:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76562432 -> 76566528 00002000:00010000:2.0:1789760777.748197:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149560 -> 149568 00010000:00010000:2.0:1789760777.748214:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748230:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748246:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748255:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.748261:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748272:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748300:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.748310:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0725 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76562432->76566527] (req 76562432->76566527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.748516:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503780864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7545us (13569us total) trans 4294985999 rc 0/0 00000100:00100000:2.0:1789760777.748529:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18703 00000100:00100000:2.0:1789760777.748542:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707846567143 time 707865559394 state 0x2 ref 0 00000100:00100000:1.0:1789760777.756048:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.761001:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503780992 00000100:00100000:0.0:1789760777.761023:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503780992 00000100:00100000:0.0:1789760777.761119:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707846567143 time 707865559394 state 0x0 ref 1 00000100:00100000:0.0:1789760777.761140:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707865819810 time 707865559394 state 0x1 ref 1 00000100:00100000:0.0:1789760777.761265:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707865819810 time 707885869666 state 0x0 ref 1 00000100:00100000:0.0:1789760777.761282:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.761351:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76566528, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24840 00010000:00010000:0.0:1789760777.761364:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.761385:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.761409:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.761418:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.761436:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.766771:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986000 is committed 00002000:00010000:0.0:1789760777.767226:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76566528 -> 76570624 00002000:00010000:0.0:1789760777.767233:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149568 -> 149576 00010000:00010000:0.0:1789760777.767252:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.767267:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.767277:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.767290:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.767296:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.767307:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.767346:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.767355:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f072c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76566528->76570623] (req 76566528->76570623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.767535:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503780992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6250us (11495us total) trans 4294986000 rc 0/0 00000100:00100000:0.0:1789760777.767548:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18704 00000100:00100000:0.0:1789760777.767561:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707865819810 time 707885869666 state 0x2 ref 0 00000100:00100000:0.0:1789760777.772978:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.773678:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503781120 00000100:00100000:1.0:1789760777.773693:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503781120 00000100:00100000:1.0:1789760777.773775:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707865819810 time 707885869666 state 0x0 ref 1 00000100:00100000:1.0:1789760777.773804:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707886130082 time 707885869666 state 0x1 ref 1 00000100:00100000:1.0:1789760777.774047:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707886130082 time 707898651757 state 0x0 ref 1 00000100:00100000:1.0:1789760777.774064:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc17850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.774131:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76570624, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24848 00010000:00010000:1.0:1789760777.774142:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.774160:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.774165:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.774173:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.774191:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.779986:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986001 is committed 00002000:00010000:1.0:1789760777.781108:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76570624 -> 76574720 00002000:00010000:1.0:1789760777.781120:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149576 -> 149584 00010000:00010000:1.0:1789760777.781137:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.781153:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.781162:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.781170:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.781176:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.781186:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.781211:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.781219:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0733 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76570624->76574719] (req 76570624->76574719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.781460:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc17850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7392us (8489us total) trans 4294986001 rc 0/0 00000100:00100000:1.0:1789760777.781472:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18705 00000100:00100000:1.0:1789760777.781485:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707886130082 time 707898651757 state 0x2 ref 0 00000100:00100000:1.0:1789760777.791200:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.791489:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503781248 00000100:00100000:2.0:1789760777.791503:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503781248 00000100:00100000:2.0:1789760777.791709:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707886130082 time 707898651757 state 0x0 ref 1 00000100:00100000:2.0:1789760777.791730:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707898912173 time 707898651757 state 0x1 ref 1 00000100:00100000:2.0:1789760777.791946:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707898912173 time 707916551067 state 0x0 ref 1 00000100:00100000:2.0:1789760777.791963:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503781248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.792029:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76574720, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x24850 00010000:00010000:2.0:1789760777.792040:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.792061:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760777.792066:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.792074:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.792091:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.799645:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986002 is committed 00002000:00010000:2.0:1789760777.799873:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76574720 -> 76578816 00002000:00010000:2.0:1789760777.799879:0:15091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:2.0:1789760777.799882:0:15091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00002000:00010000:2.0:1789760777.799885:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149584 -> 149592 00010000:00010000:2.0:1789760777.799904:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.799920:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.799930:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.799939:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.799945:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.799956:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.799981:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.799990:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f073a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76574720->76578815] (req 76574720->76578815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.800119:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503781248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8152us (8928us total) trans 4294986002 rc 0/0 00000100:00100000:2.0:1789760777.800132:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18706 00000100:00100000:2.0:1789760777.800145:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707898912173 time 707916551067 state 0x2 ref 0 00000100:00100000:2.0:1789760777.804685:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.804761:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503781376 00000100:00100000:0.0:1789760777.804773:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503781376 00000100:00100000:0.0:1789760777.804897:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707898912173 time 707916551067 state 0x0 ref 1 00000100:00100000:0.0:1789760777.804915:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707916811483 time 707916551067 state 0x1 ref 1 00000100:00100000:0.0:1789760777.804953:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707916811483 time 707929559213 state 0x0 ref 1 00000100:00100000:0.0:1789760777.804969:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503781376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.805037:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76578816, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24858 00010000:00010000:0.0:1789760777.805049:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.805069:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.805074:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.805081:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.805097:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.815518:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986003 is committed 00002000:00010000:0.0:1789760777.817719:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76578816 -> 76582912 00002000:00010000:0.0:1789760777.817731:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149592 -> 149600 00010000:00010000:0.0:1789760777.817748:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817764:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817774:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817782:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.817789:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817802:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817828:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817836:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0741 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76578816->76582911] (req 76578816->76582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.817971:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503781376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12999us (13296us total) trans 4294986003 rc 0/0 00000100:00100000:0.0:1789760777.817984:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18707 00000100:00100000:0.0:1789760777.817996:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707916811483 time 707929559213 state 0x2 ref 0 00000100:00100000:0.0:1789760777.822210:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.822591:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503781504 00000100:00100000:1.0:1789760777.822616:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503781504 00000100:00100000:0.0:1789760777.822776:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707916811483 time 707929559213 state 0x0 ref 1 00000100:00100000:0.0:1789760777.822794:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707929819629 time 707929559213 state 0x1 ref 1 00000100:00100000:0.0:1789760777.822870:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707929819629 time 707947475208 state 0x0 ref 1 00000100:00100000:0.0:1789760777.822885:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503781504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.822950:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76582912, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24860 00010000:00010000:0.0:1789760777.822962:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.822983:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760777.822988:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.822996:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.823013:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.829478:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986004 is committed 00002000:00010000:0.0:1789760777.829689:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76582912 -> 76587008 00002000:00010000:0.0:1789760777.829696:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149600 -> 149608 00010000:00010000:0.0:1789760777.829716:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.829732:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.829743:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.829751:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.829757:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.829768:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.829797:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.829806:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0748 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76582912->76587007] (req 76582912->76587007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.830000:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503781504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7109us (7798us total) trans 4294986004 rc 0/0 00000100:00100000:0.0:1789760777.830012:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18708 00000100:00100000:0.0:1789760777.830026:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707929819629 time 707947475208 state 0x2 ref 0 00000100:00100000:1.0:1789760777.835266:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760777.837210:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503781632 00000100:00100000:3.0:1789760777.837232:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503781632 00000100:00100000:3.0:1789760777.837334:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707929819629 time 707947475208 state 0x0 ref 1 00000100:00100000:3.0:1789760777.837354:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707947735624 time 707947475208 state 0x1 ref 1 00000100:00100000:3.0:1789760777.837417:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707947735624 time 707962022641 state 0x0 ref 1 00000100:00100000:3.0:1789760777.837433:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760777.837504:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76587008, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24868 00010000:00010000:3.0:1789760777.837515:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.837536:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760777.837542:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.837550:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.837567:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.843620:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986005 is committed 00002000:00010000:2.0:1789760777.844278:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76587008 -> 76591104 00002000:00010000:2.0:1789760777.844285:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149608 -> 149616 00010000:00010000:2.0:1789760777.844303:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.844329:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.844341:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.844349:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.844354:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.844364:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.845048:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.845064:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f074f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76587008->76591103] (req 76587008->76591103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.845258:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7822us (9999us total) trans 4294986005 rc 0/0 00000100:00100000:2.0:1789760777.845272:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18709 00000100:00100000:2.0:1789760777.845286:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707947735624 time 707962022641 state 0x2 ref 0 00000100:00100000:2.0:1789760777.853080:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.853197:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503781760 00000100:00100000:1.0:1789760777.853210:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503781760 00000100:00100000:1.0:1789760777.853283:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707947735624 time 707962022641 state 0x0 ref 1 00000100:00100000:1.0:1789760777.853302:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707962283057 time 707962022641 state 0x1 ref 1 00000100:00100000:1.0:1789760777.853347:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707962283057 time 707977952848 state 0x0 ref 1 00000100:00100000:1.0:1789760777.853361:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.853479:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76591104, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24870 00010000:00010000:1.0:1789760777.853490:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.853509:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.853515:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.853523:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.853539:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.860626:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986006 is committed 00002000:00010000:1.0:1789760777.861260:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76591104 -> 76595200 00002000:00010000:1.0:1789760777.861267:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149616 -> 149624 00010000:00010000:1.0:1789760777.861295:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.861311:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.861530:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.861540:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.861546:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.861558:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.861584:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.861593:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0756 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76591104->76595199] (req 76591104->76595199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.861725:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8359us (8653us total) trans 4294986006 rc 0/0 00000100:00100000:1.0:1789760777.861739:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18710 00000100:00100000:1.0:1789760777.861752:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707962283057 time 707977952848 state 0x2 ref 0 00000100:00100000:0.0:1789760777.869113:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.869483:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503781888 00000100:00100000:1.0:1789760777.869497:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503781888 00000100:00100000:1.0:1789760777.869566:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707962283057 time 707977952848 state 0x0 ref 1 00000100:00100000:1.0:1789760777.869584:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707978213264 time 707977952848 state 0x1 ref 1 00000100:00100000:1.0:1789760777.869623:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707978213264 time 707994229410 state 0x0 ref 1 00000100:00100000:1.0:1789760777.869637:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.871113:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76595200, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24878 00010000:00010000:1.0:1789760777.871128:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.871148:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760777.871153:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.871160:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.871179:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.882457:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986007 is committed 00002000:00010000:1.0:1789760777.882708:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76595200 -> 76599296 00002000:00010000:1.0:1789760777.882714:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149624 -> 149632 00010000:00010000:1.0:1789760777.882732:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.882747:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.882756:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.882766:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.882771:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.882782:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.882809:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.882818:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f075d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76595200->76599295] (req 76595200->76599295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.882940:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503781888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13299us (13836us total) trans 4294986007 rc 0/0 00000100:00100000:1.0:1789760777.882951:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18711 00000100:00100000:1.0:1789760777.882963:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707978213264 time 707994229410 state 0x2 ref 0 00000100:00100000:0.0:1789760777.895976:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.896381:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782016 00000100:00100000:2.0:1789760777.896424:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782016 00000100:00100000:2.0:1789760777.896496:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707978213264 time 707994229410 state 0x0 ref 1 00000100:00100000:2.0:1789760777.896515:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707994489826 time 707994229410 state 0x1 ref 1 00000100:00100000:2.0:1789760777.896547:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707994489826 time 708021154333 state 0x0 ref 1 00000100:00100000:2.0:1789760777.896559:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.896629:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76599296, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24880 00010000:00010000:2.0:1789760777.896643:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.896663:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760777.896668:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.896676:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.896692:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.902847:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986008 is committed 00002000:00010000:2.0:1789760777.902984:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76599296 -> 76603392 00002000:00010000:2.0:1789760777.902990:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149632 -> 149640 00010000:00010000:2.0:1789760777.903006:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903020:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903030:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903038:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.903044:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903055:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903079:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.903087:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0764 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76599296->76603391] (req 76599296->76603391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.903261:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6695us (7293us total) trans 4294986008 rc 0/0 00000100:00100000:2.0:1789760777.903286:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18712 00000100:00100000:2.0:1789760777.903300:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707994489826 time 708021154333 state 0x2 ref 0 00000100:00100000:2.0:1789760777.910733:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.913871:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782144 00000100:00100000:1.0:1789760777.914035:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782144 00000100:00100000:1.0:1789760777.914130:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 707994489826 time 708021154333 state 0x0 ref 1 00000100:00100000:1.0:1789760777.914150:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708021414749 time 708021154333 state 0x1 ref 1 00000100:00100000:1.0:1789760777.914296:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708021414749 time 708038900511 state 0x0 ref 1 00000100:00100000:1.0:1789760777.914314:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.914425:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76603392, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24888 00010000:00010000:1.0:1789760777.914437:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.914458:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760777.914464:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.914472:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.914489:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.921208:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986009 is committed 00002000:00010000:1.0:1789760777.922586:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76603392 -> 76607488 00002000:00010000:1.0:1789760777.922607:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149640 -> 149648 00010000:00010000:1.0:1789760777.922626:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.922647:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.922657:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.922665:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.922671:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.922681:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.922711:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.922719:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f076b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76603392->76607487] (req 76603392->76607487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.922852:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8535us (12127us total) trans 4294986009 rc 0/0 00000100:00100000:1.0:1789760777.922865:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18713 00000100:00100000:1.0:1789760777.922878:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708021414749 time 708038900511 state 0x2 ref 0 00000100:00100000:0.0:1789760777.929533:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.929676:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782272 00000100:00100000:1.0:1789760777.929690:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782272 00000100:00100000:1.0:1789760777.929770:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708021414749 time 708038900511 state 0x0 ref 1 00000100:00100000:1.0:1789760777.929789:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708039160927 time 708038900511 state 0x1 ref 1 00000100:00100000:1.0:1789760777.929827:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708039160927 time 708054433272 state 0x0 ref 1 00000100:00100000:1.0:1789760777.929841:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.929904:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76607488, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24890 00010000:00010000:1.0:1789760777.929917:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.929939:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760777.929944:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.929952:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.929970:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.939977:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986010 is committed 00002000:00010000:1.0:1789760777.940178:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76607488 -> 76611584 00002000:00010000:1.0:1789760777.940184:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149648 -> 149656 00010000:00010000:1.0:1789760777.940202:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940218:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940235:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940244:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.940249:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940259:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940284:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.940293:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0772 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76607488->76611583] (req 76607488->76611583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.941105:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11253us (11576us total) trans 4294986010 rc 0/0 00000100:00100000:1.0:1789760777.941124:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18714 00000100:00100000:1.0:1789760777.941136:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708039160927 time 708054433272 state 0x2 ref 0 00000100:00100000:0.0:1789760777.947043:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760777.947144:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782400 00000100:00100000:1.0:1789760777.947157:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782400 00000100:00100000:1.0:1789760777.947234:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708039160927 time 708054433272 state 0x0 ref 1 00000100:00100000:1.0:1789760777.947253:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708054693688 time 708054433272 state 0x1 ref 1 00000100:00100000:1.0:1789760777.947292:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708054693688 time 708071898599 state 0x0 ref 1 00000100:00100000:1.0:1789760777.947306:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760777.947382:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76611584, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24898 00010000:00010000:1.0:1789760777.947434:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.947454:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760777.947458:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.947466:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.947483:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.953628:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986011 is committed 00002000:00010000:1.0:1789760777.953974:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76611584 -> 76615680 00002000:00010000:1.0:1789760777.953981:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149656 -> 149664 00010000:00010000:1.0:1789760777.953998:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.954014:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.954025:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.954040:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760777.954047:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.954057:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.954083:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.954091:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0779 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76611584->76615679] (req 76611584->76615679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.954256:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6945us (7221us total) trans 4294986011 rc 0/0 00000100:00100000:1.0:1789760777.954269:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18715 00000100:00100000:1.0:1789760777.954282:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708054693688 time 708071898599 state 0x2 ref 0 00000100:00100000:2.0:1789760777.960313:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760777.960530:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782528 00000100:00100000:0.0:1789760777.960543:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782528 00000100:00100000:0.0:1789760777.960627:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708054693688 time 708071898599 state 0x0 ref 1 00000100:00100000:0.0:1789760777.961871:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708072159015 time 708071898599 state 0x1 ref 1 00000100:00100000:0.0:1789760777.961930:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708072159015 time 708086535301 state 0x0 ref 1 00000100:00100000:0.0:1789760777.961955:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503782528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760777.962026:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76615680, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248a0 00010000:00010000:0.0:1789760777.962040:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.962061:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760777.962066:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.962074:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.962090:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.968234:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986012 is committed 00002000:00010000:0.0:1789760777.968856:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76615680 -> 76619776 00002000:00010000:0.0:1789760777.969299:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149664 -> 149672 00010000:00010000:0.0:1789760777.969316:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.969328:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.969334:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.969339:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760777.969344:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.969355:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.969381:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.969641:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0780 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76615680->76619775] (req 76615680->76619775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.970038:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503782528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8083us (9729us total) trans 4294986012 rc 0/0 00000100:00100000:0.0:1789760777.970050:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18716 00000100:00100000:0.0:1789760777.970064:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708072159015 time 708086535301 state 0x2 ref 0 00000100:00100000:0.0:1789760777.976932:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.977039:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782656 00000100:00100000:2.0:1789760777.977053:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782656 00000100:00100000:2.0:1789760777.977130:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708072159015 time 708086535301 state 0x0 ref 1 00000100:00100000:2.0:1789760777.977151:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708086795717 time 708086535301 state 0x1 ref 1 00000100:00100000:2.0:1789760777.977183:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708086795717 time 708101788534 state 0x0 ref 1 00000100:00100000:2.0:1789760777.977198:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503782656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.977263:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76619776, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248a8 00010000:00010000:2.0:1789760777.977275:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.977297:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760777.977303:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.977310:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.977327:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.986717:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986013 is committed 00002000:00010000:2.0:1789760777.987126:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76619776 -> 76623872 00002000:00010000:2.0:1789760777.987132:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149672 -> 149680 00010000:00010000:2.0:1789760777.987150:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.987169:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.987178:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.987185:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760777.987190:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.987200:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.987225:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.987235:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0787 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76619776->76623871] (req 76619776->76623871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.987637:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503782656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10435us (10713us total) trans 4294986013 rc 0/0 00000100:00100000:2.0:1789760777.987650:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18717 00000100:00100000:2.0:1789760777.987662:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708086795717 time 708101788534 state 0x2 ref 0 00000100:00100000:1.0:1789760777.996772:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760777.997830:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782784 00000100:00100000:2.0:1789760777.997849:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782784 00000100:00100000:2.0:1789760777.998285:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708086795717 time 708101788534 state 0x0 ref 1 00000100:00100000:2.0:1789760777.998306:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708102048950 time 708101788534 state 0x1 ref 1 00000100:00100000:2.0:1789760777.998511:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708102048950 time 708123116203 state 0x0 ref 1 00000100:00100000:2.0:1789760777.998531:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760777.998609:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76623872, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248b0 00010000:00010000:2.0:1789760777.998621:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.998773:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760777.998779:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.998787:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.998805:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.006183:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986014 is committed 00002000:00010000:2.0:1789760778.006801:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76623872 -> 76627968 00002000:00010000:2.0:1789760778.006807:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149680 -> 149688 00010000:00010000:2.0:1789760778.006821:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.006835:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.006849:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.006857:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.006862:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.006872:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.006893:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.006900:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f078e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76623872->76627967] (req 76623872->76627967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.007031:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503782784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8501us (10270us total) trans 4294986014 rc 0/0 00000100:00100000:2.0:1789760778.007041:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18718 00000100:00100000:2.0:1789760778.007051:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708102048950 time 708123116203 state 0x2 ref 0 00000100:00100000:3.0:1789760778.014589:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.014813:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503782912 00000100:00100000:1.0:1789760778.014826:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503782912 00000100:00100000:1.0:1789760778.014935:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708102048950 time 708123116203 state 0x0 ref 1 00000100:00100000:1.0:1789760778.014955:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708123376619 time 708123116203 state 0x1 ref 1 00000100:00100000:1.0:1789760778.015001:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708123376619 time 708139606760 state 0x0 ref 1 00000100:00100000:1.0:1789760778.015018:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503782912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.015088:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76627968, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248b8 00010000:00010000:1.0:1789760778.015099:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.015120:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.015126:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.015134:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.015152:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760778.016426:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760778.025326:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986015 is committed 00002000:00010000:1.0:1789760778.025507:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76627968 -> 76632064 00002000:00010000:1.0:1789760778.025514:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149688 -> 149696 00010000:00010000:1.0:1789760778.025531:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.025546:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.025556:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.025564:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.025572:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.025582:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.025617:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.025626:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0795 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76627968->76632063] (req 76627968->76632063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.025829:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503782912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10805us (11248us total) trans 4294986015 rc 0/0 00000100:00100000:1.0:1789760778.025844:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18719 00000100:00100000:1.0:1789760778.025858:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708123376619 time 708139606760 state 0x2 ref 0 00000100:00100000:0.0:1789760778.032679:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.032808:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783040 00000100:00100000:2.0:1789760778.032822:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783040 00000100:00100000:2.0:1789760778.032897:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708123376619 time 708139606760 state 0x0 ref 1 00000100:00100000:2.0:1789760778.032932:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708139867176 time 708139606760 state 0x1 ref 1 00000100:00100000:2.0:1789760778.033027:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708139867176 time 708157632300 state 0x0 ref 1 00000100:00100000:2.0:1789760778.033042:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503783040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.035240:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76632064, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248c0 00010000:00010000:2.0:1789760778.035264:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.035287:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.035292:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.035302:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.035330:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.041155:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986016 is committed 00002000:00010000:2.0:1789760778.041299:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76632064 -> 76636160 00002000:00010000:2.0:1789760778.041305:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149696 -> 149704 00010000:00010000:2.0:1789760778.041331:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.041346:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.041355:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.041364:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.041369:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.041379:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.041438:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.041447:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f079c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76632064->76636159] (req 76632064->76636159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.041931:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503783040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8883us (9261us total) trans 4294986016 rc 0/0 00000100:00100000:2.0:1789760778.041944:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18720 00000100:00100000:2.0:1789760778.041957:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708139867176 time 708157632300 state 0x2 ref 0 00000100:00100000:0.0:1789760778.060349:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.060550:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783168 00000100:00100000:2.0:1789760778.060565:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783168 00000100:00100000:2.0:1789760778.060654:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708139867176 time 708157632300 state 0x0 ref 1 00000100:00100000:2.0:1789760778.060673:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708157892716 time 708157632300 state 0x1 ref 1 00000100:00100000:2.0:1789760778.060713:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708157892716 time 708185318669 state 0x0 ref 1 00000100:00100000:2.0:1789760778.060729:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503783168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.060793:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76636160, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248c8 00010000:00010000:2.0:1789760778.060804:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.060825:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.060830:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.060837:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.060851:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.069494:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986017 is committed 00002000:00010000:2.0:1789760778.069729:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76636160 -> 76640256 00002000:00010000:2.0:1789760778.069734:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149704 -> 149712 00010000:00010000:2.0:1789760778.069753:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069769:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069779:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069786:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.069792:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069802:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069826:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069834:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76636160->76640255] (req 76636160->76640255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.070095:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503783168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9359us (9765us total) trans 4294986017 rc 0/0 00000100:00100000:2.0:1789760778.070108:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18721 00000100:00100000:2.0:1789760778.070121:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708157892716 time 708185318669 state 0x2 ref 0 00000100:00100000:3.0:1789760778.076466:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.077000:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783296 00000100:00100000:1.0:1789760778.077017:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783296 00000100:00100000:1.0:1789760778.077099:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708157892716 time 708185318669 state 0x0 ref 1 00000100:00100000:1.0:1789760778.077120:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708185579085 time 708185318669 state 0x1 ref 1 00000100:00100000:1.0:1789760778.077149:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708185579085 time 708201755484 state 0x0 ref 1 00000100:00100000:1.0:1789760778.077163:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503783296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.077232:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76640256, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248d0 00010000:00010000:1.0:1789760778.077244:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.077264:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.077268:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.077275:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.077291:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.085377:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986018 is committed 00002000:00010000:1.0:1789760778.085731:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76640256 -> 76644352 00002000:00010000:1.0:1789760778.085738:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149712 -> 149720 00010000:00010000:1.0:1789760778.085756:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.085772:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.085781:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.085790:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.085796:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.085805:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.085832:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.085840:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76640256->76644351] (req 76640256->76644351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.086004:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503783296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8835us (9546us total) trans 4294986018 rc 0/0 00000100:00100000:1.0:1789760778.086017:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18722 00000100:00100000:1.0:1789760778.086029:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708185579085 time 708201755484 state 0x2 ref 0 00000100:00100000:3.0:1789760778.090186:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.090301:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783424 00000100:00100000:1.0:1789760778.090317:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783424 00000100:00100000:1.0:1789760778.090740:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708185579085 time 708201755484 state 0x0 ref 1 00000100:00100000:1.0:1789760778.090760:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708202015900 time 708201755484 state 0x1 ref 1 00000100:00100000:1.0:1789760778.090837:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708202015900 time 708215441689 state 0x0 ref 1 00000100:00100000:1.0:1789760778.090852:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503783424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.090921:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76644352, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248d8 00010000:00010000:1.0:1789760778.090933:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.090953:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.090959:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.090968:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.090986:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.096201:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986019 is committed 00002000:00010000:1.0:1789760778.096340:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76644352 -> 76648448 00002000:00010000:1.0:1789760778.096347:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149720 -> 149728 00010000:00010000:1.0:1789760778.096364:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.096380:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.096411:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.096421:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.096426:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.096437:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.096462:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.096475:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76644352->76648447] (req 76644352->76648447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.096612:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503783424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5745us (6425us total) trans 4294986019 rc 0/0 00000100:00100000:1.0:1789760778.096625:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18723 00000100:00100000:1.0:1789760778.098582:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708202015900 time 708215441689 state 0x2 ref 0 00000100:00100000:0.0:1789760778.101903:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.102029:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783552 00000100:00100000:2.0:1789760778.102042:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783552 00000100:00100000:2.0:1789760778.102115:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708202015900 time 708215441689 state 0x0 ref 1 00000100:00100000:2.0:1789760778.102134:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708215702105 time 708215441689 state 0x1 ref 1 00000100:00100000:2.0:1789760778.102166:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708215702105 time 708226772348 state 0x0 ref 1 00000100:00100000:2.0:1789760778.102179:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503783552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.102243:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76648448, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248e0 00010000:00010000:2.0:1789760778.102255:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.102276:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.102282:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.102289:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.102306:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.111463:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986020 is committed 00002000:00010000:0.0:1789760778.111602:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76648448 -> 76652544 00002000:00010000:0.0:1789760778.111609:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149728 -> 149736 00010000:00010000:0.0:1789760778.111627:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.111720:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.111730:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.111738:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.111744:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.111755:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.111782:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.111791:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f07b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76648448->76652543] (req 76648448->76652543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.111992:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503783552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9806us (10096us total) trans 4294986020 rc 0/0 00000100:00100000:0.0:1789760778.112006:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18724 00000100:00100000:0.0:1789760778.112021:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708215702105 time 708226772348 state 0x2 ref 0 00000100:00100000:3.0:1789760778.118944:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.119182:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783680 00000100:00100000:2.0:1789760778.119195:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783680 00000100:00100000:2.0:1789760778.119272:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708215702105 time 708226772348 state 0x0 ref 1 00000100:00100000:2.0:1789760778.119292:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708227032764 time 708226772348 state 0x1 ref 1 00000100:00100000:1.0:1789760778.119536:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708227032764 time 708244138173 state 0x0 ref 1 00000100:00100000:1.0:1789760778.119555:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503783680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.119628:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76652544, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248e8 00010000:00010000:1.0:1789760778.119651:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.119671:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.119677:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.119685:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.119702:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.123841:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986021 is committed 00002000:00010000:1.0:1789760778.124057:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76652544 -> 76656640 00002000:00010000:1.0:1789760778.124064:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149736 -> 149744 00010000:00010000:1.0:1789760778.124190:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.124207:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.124216:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.124224:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.124229:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.124240:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.124264:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.124271:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76652544->76656639] (req 76652544->76656639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.124552:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503783680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4994us (5616us total) trans 4294986021 rc 0/0 00000100:00100000:1.0:1789760778.124564:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18725 00000100:00100000:1.0:1789760778.124577:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708227032764 time 708244138173 state 0x2 ref 0 00000100:00100000:3.0:1789760778.128972:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.129091:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783808 00000100:00100000:1.0:1789760778.129105:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783808 00000100:00100000:1.0:1789760778.129179:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708227032764 time 708244138173 state 0x0 ref 1 00000100:00100000:1.0:1789760778.129197:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708244398589 time 708244138173 state 0x1 ref 1 00000100:00100000:1.0:1789760778.129227:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708244398589 time 708253832993 state 0x0 ref 1 00000100:00100000:1.0:1789760778.129243:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503783808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.129308:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76656640, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248f0 00010000:00010000:1.0:1789760778.129325:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.129346:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.129352:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.129360:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.129377:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.137559:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986022 is committed 00002000:00010000:1.0:1789760778.137886:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76656640 -> 76660736 00002000:00010000:1.0:1789760778.137893:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149744 -> 149752 00010000:00010000:1.0:1789760778.137912:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.137935:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.137945:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.137954:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.137959:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.137971:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.137995:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.138003:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f07c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76656640->76660735] (req 76656640->76660735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.138176:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503783808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8928us (9212us total) trans 4294986022 rc 0/0 00000100:00100000:1.0:1789760778.138188:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18726 00000100:00100000:1.0:1789760778.138201:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708244398589 time 708253832993 state 0x2 ref 0 00000100:00100000:0.0:1789760778.141380:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.141540:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503783936 00000100:00100000:2.0:1789760778.141554:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503783936 00000100:00100000:2.0:1789760778.141707:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708244398589 time 708253832993 state 0x0 ref 1 00000100:00100000:2.0:1789760778.141728:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708254093409 time 708253832993 state 0x1 ref 1 00000100:00100000:2.0:1789760778.141756:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708254093409 time 708266362235 state 0x0 ref 1 00000100:00100000:2.0:1789760778.141771:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503783936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.141848:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76660736, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x248f8 00010000:00010000:2.0:1789760778.141859:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.141878:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.141884:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.141892:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.141908:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.151179:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986023 is committed 00002000:00010000:2.0:1789760778.151511:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76660736 -> 76664832 00002000:00010000:2.0:1789760778.151518:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149752 -> 149760 00010000:00010000:2.0:1789760778.151537:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.151552:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.151562:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.151570:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.151576:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.151591:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.151626:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.151643:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76660736->76664831] (req 76660736->76664831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.151776:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503783936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10001us (10405us total) trans 4294986023 rc 0/0 00000100:00100000:2.0:1789760778.151789:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18727 00000100:00100000:2.0:1789760778.151802:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708254093409 time 708266362235 state 0x2 ref 0 00000100:00100000:3.0:1789760778.158706:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.159071:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784064 00000100:00100000:0.0:1789760778.159084:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784064 00000100:00100000:0.0:1789760778.159169:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708254093409 time 708266362235 state 0x0 ref 1 00000100:00100000:0.0:1789760778.159188:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708266622651 time 708266362235 state 0x1 ref 1 00000100:00100000:0.0:1789760778.159228:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708266622651 time 708283833993 state 0x0 ref 1 00000100:00100000:0.0:1789760778.159243:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503784064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.159306:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76664832, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24900 00010000:00010000:0.0:1789760778.159328:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.159347:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.159352:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.159360:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.159375:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.166624:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986024 is committed 00002000:00010000:0.0:1789760778.166764:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76664832 -> 76668928 00002000:00010000:0.0:1789760778.166772:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149760 -> 149768 00010000:00010000:0.0:1789760778.166790:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.166808:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.166819:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.166828:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.166834:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.166845:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.166869:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.166878:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76664832->76668927] (req 76664832->76668927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.167092:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503784064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7844us (8394us total) trans 4294986024 rc 0/0 00000100:00100000:0.0:1789760778.167105:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18728 00000100:00100000:0.0:1789760778.167117:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708266622651 time 708283833993 state 0x2 ref 0 00000100:00100000:3.0:1789760778.174911:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.175179:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784192 00000100:00100000:1.0:1789760778.175191:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784192 00000100:00100000:1.0:1789760778.175256:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708266622651 time 708283833993 state 0x0 ref 1 00000100:00100000:1.0:1789760778.175272:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708284094409 time 708283833993 state 0x1 ref 1 00000100:00100000:1.0:1789760778.175299:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708284094409 time 708299906929 state 0x0 ref 1 00000100:00100000:1.0:1789760778.175310:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503784192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.175368:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76668928, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24908 00010000:00010000:1.0:1789760778.175376:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.175433:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.175438:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.175444:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.175458:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.184526:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986025 is committed 00002000:00010000:0.0:1789760778.184740:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76668928 -> 76673024 00002000:00010000:0.0:1789760778.184748:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149768 -> 149776 00010000:00010000:0.0:1789760778.184767:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.184855:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.184867:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.184876:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.184881:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.184891:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.184918:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.184926:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f07db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76668928->76673023] (req 76668928->76673023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.185122:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503784192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9804us (10219us total) trans 4294986025 rc 0/0 00000100:00100000:0.0:1789760778.185134:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18729 00000100:00100000:0.0:1789760778.185147:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708284094409 time 708299906929 state 0x2 ref 0 00000100:00100000:1.0:1789760778.189206:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.190178:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784320 00000100:00100000:2.0:1789760778.190190:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784320 00000100:00100000:2.0:1789760778.190251:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708284094409 time 708299906929 state 0x0 ref 1 00000100:00100000:2.0:1789760778.190272:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708300167345 time 708299906929 state 0x1 ref 1 00000100:00100000:2.0:1789760778.190304:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708300167345 time 708314909843 state 0x0 ref 1 00000100:00100000:2.0:1789760778.190317:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503784320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.190374:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76673024, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24910 00010000:00010000:2.0:1789760778.190383:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.190421:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.190426:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.190433:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.190449:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.196218:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986026 is committed 00002000:00010000:2.0:1789760778.198570:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76673024 -> 76677120 00002000:00010000:2.0:1789760778.198581:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149776 -> 149784 00010000:00010000:2.0:1789760778.198604:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.198618:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.198628:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.198644:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.198649:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.198659:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.198685:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.198693:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f07e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76673024->76677119] (req 76673024->76677119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.198791:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503784320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8472us (9589us total) trans 4294986026 rc 0/0 00000100:00100000:2.0:1789760778.198801:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18730 00000100:00100000:2.0:1789760778.198812:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708300167345 time 708314909843 state 0x2 ref 0 00000100:00100000:3.0:1789760778.201700:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.201786:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784448 00000100:00100000:2.0:1789760778.201797:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784448 00000100:00100000:2.0:1789760778.201861:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708300167345 time 708314909843 state 0x0 ref 1 00000100:00100000:2.0:1789760778.201879:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708315170259 time 708314909843 state 0x1 ref 1 00000100:00100000:2.0:1789760778.201913:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708315170259 time 708326518718 state 0x0 ref 1 00000100:00100000:2.0:1789760778.201925:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503784448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.201974:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76677120, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24918 00010000:00010000:2.0:1789760778.201983:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.201999:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.202005:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.202012:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.202028:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.209007:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986027 is committed 00002000:00010000:2.0:1789760778.209192:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76677120 -> 76681216 00002000:00010000:2.0:1789760778.209197:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149784 -> 149792 00010000:00010000:2.0:1789760778.209209:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.209221:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.209607:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.209615:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.209620:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.209630:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.209651:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.209664:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f07e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76677120->76681215] (req 76677120->76681215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.209765:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503784448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7838us (8070us total) trans 4294986027 rc 0/0 00000100:00100000:2.0:1789760778.209776:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18731 00000100:00100000:2.0:1789760778.209787:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708315170259 time 708326518718 state 0x2 ref 0 00000100:00100000:3.0:1789760778.213757:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.217064:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784576 00000100:00100000:0.0:1789760778.217096:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784576 00000100:00100000:0.0:1789760778.217192:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708315170259 time 708326518718 state 0x0 ref 1 00000100:00100000:0.0:1789760778.217213:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708326779134 time 708326518718 state 0x1 ref 1 00000100:00100000:0.0:1789760778.217334:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708326779134 time 708341939086 state 0x0 ref 1 00000100:00100000:0.0:1789760778.217352:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503784576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.217469:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76681216, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24920 00010000:00010000:0.0:1789760778.217482:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217503:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.217509:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.217517:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217535:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.225431:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986028 is committed 00002000:00010000:0.0:1789760778.229766:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76681216 -> 76685312 00002000:00010000:0.0:1789760778.229781:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149792 -> 149800 00010000:00010000:0.0:1789760778.229800:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.229816:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.229831:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.229841:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.229847:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.229858:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.229883:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.229892:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f07f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76681216->76685311] (req 76681216->76685311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.230046:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503784576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12691us (16298us total) trans 4294986028 rc 0/0 00000100:00100000:0.0:1789760778.230060:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18732 00000100:00100000:0.0:1789760778.230073:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708326779134 time 708341939086 state 0x2 ref 0 00000100:00100000:1.0:1789760778.237230:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.239176:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784704 00000100:00100000:0.0:1789760778.239198:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784704 00000100:00100000:0.0:1789760778.239289:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708326779134 time 708341939086 state 0x0 ref 1 00000100:00100000:0.0:1789760778.239309:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708342199502 time 708341939086 state 0x1 ref 1 00000100:00100000:0.0:1789760778.239352:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708342199502 time 708363958449 state 0x0 ref 1 00000100:00100000:0.0:1789760778.239369:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503784704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.239454:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76685312, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24928 00010000:00010000:0.0:1789760778.239466:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.239488:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.239493:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.239507:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.239524:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.244343:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986029 is committed 00002000:00010000:0.0:1789760778.244583:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76685312 -> 76689408 00002000:00010000:0.0:1789760778.244589:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149800 -> 149808 00010000:00010000:0.0:1789760778.244617:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.244632:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.244641:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.244655:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.244661:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.244672:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.244698:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.244708:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f07f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76685312->76689407] (req 76685312->76689407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.244824:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503784704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5452us (7600us total) trans 4294986029 rc 0/0 00000100:00100000:0.0:1789760778.244836:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18733 00000100:00100000:0.0:1789760778.244850:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708342199502 time 708363958449 state 0x2 ref 0 00000100:00100000:3.0:1789760778.250861:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.251034:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784832 00000100:00100000:1.0:1789760778.251045:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784832 00000100:00100000:1.0:1789760778.251133:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708342199502 time 708363958449 state 0x0 ref 1 00000100:00100000:1.0:1789760778.251155:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708364218865 time 708363958449 state 0x1 ref 1 00000100:00100000:1.0:1789760778.251192:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708364218865 time 708375797855 state 0x0 ref 1 00000100:00100000:1.0:1789760778.251214:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503784832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.251274:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76689408, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24930 00010000:00010000:1.0:1789760778.251284:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.251303:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.251308:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.251315:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.251335:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.258550:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986030 is committed 00002000:00010000:2.0:1789760778.260626:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76689408 -> 76693504 00002000:00010000:2.0:1789760778.260700:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149808 -> 149816 00010000:00010000:2.0:1789760778.260804:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.260822:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.260833:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.260841:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.260847:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.260857:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.261046:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.261055:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f07fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76689408->76693503] (req 76689408->76693503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.261233:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503784832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10019us (10376us total) trans 4294986030 rc 0/0 00000100:00100000:2.0:1789760778.261245:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18734 00000100:00100000:2.0:1789760778.261258:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708364218865 time 708375797855 state 0x2 ref 0 00000100:00100000:3.0:1789760778.266620:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.266982:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503784960 00000100:00100000:2.0:1789760778.267000:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503784960 00000100:00100000:2.0:1789760778.267077:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708364218865 time 708375797855 state 0x0 ref 1 00000100:00100000:2.0:1789760778.267097:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708376058271 time 708375797855 state 0x1 ref 1 00000100:00100000:2.0:1789760778.267141:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708376058271 time 708391746958 state 0x0 ref 1 00000100:00100000:2.0:1789760778.267156:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503784960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.267219:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76693504, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24938 00010000:00010000:2.0:1789760778.267230:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.267250:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.267257:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.267265:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.267282:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.272867:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986031 is committed 00002000:00010000:2.0:1789760778.273176:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76693504 -> 76697600 00002000:00010000:2.0:1789760778.273183:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149816 -> 149824 00010000:00010000:2.0:1789760778.273202:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.273217:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.273228:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.273237:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.273242:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.273253:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.273278:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.273293:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0805 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76693504->76697599] (req 76693504->76697599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.273619:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503784960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6457us (7004us total) trans 4294986031 rc 0/0 00000100:00100000:2.0:1789760778.273633:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18735 00000100:00100000:2.0:1789760778.273646:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708376058271 time 708391746958 state 0x2 ref 0 00000100:00100000:1.0:1789760778.282768:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.282904:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785088 00000100:00100000:0.0:1789760778.282919:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785088 00000100:00100000:0.0:1789760778.283341:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708376058271 time 708391746958 state 0x0 ref 1 00000100:00100000:0.0:1789760778.283361:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708392007374 time 708391746958 state 0x1 ref 1 00000100:00100000:0.0:1789760778.283492:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708392007374 time 708408097387 state 0x0 ref 1 00000100:00100000:0.0:1789760778.283513:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.283577:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76697600, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24940 00010000:00010000:0.0:1789760778.283588:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.283623:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.283628:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.283636:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.283654:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.289199:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986032 is committed 00002000:00010000:0.0:1789760778.289471:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76697600 -> 76701696 00002000:00010000:0.0:1789760778.289478:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149824 -> 149832 00010000:00010000:0.0:1789760778.289496:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.289510:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.289520:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.289528:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.289535:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.289546:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.289571:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.289579:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f080c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76697600->76701695] (req 76697600->76701695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.289737:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6221us (7097us total) trans 4294986032 rc 0/0 00000100:00100000:0.0:1789760778.289749:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18736 00000100:00100000:0.0:1789760778.289761:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708392007374 time 708408097387 state 0x2 ref 0 00000100:00100000:3.0:1789760778.293423:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.293483:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785216 00000100:00100000:0.0:1789760778.293496:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785216 00000100:00100000:0.0:1789760778.293571:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708392007374 time 708408097387 state 0x0 ref 1 00000100:00100000:0.0:1789760778.293589:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708408357803 time 708408097387 state 0x1 ref 1 00000100:00100000:0.0:1789760778.293644:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708408357803 time 708418242363 state 0x0 ref 1 00000100:00100000:0.0:1789760778.293668:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.293733:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76701696, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24948 00010000:00010000:0.0:1789760778.293746:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.293767:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.293773:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.293780:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.293797:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.297938:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986033 is committed 00002000:00010000:0.0:1789760778.298107:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76701696 -> 76705792 00002000:00010000:0.0:1789760778.298114:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149832 -> 149840 00010000:00010000:0.0:1789760778.298130:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.298146:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.298156:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.298165:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.298171:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.298181:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.298210:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.298219:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0813 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76701696->76705791] (req 76701696->76705791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.298507:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4833us (5113us total) trans 4294986033 rc 0/0 00000100:00100000:0.0:1789760778.298520:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18737 00000100:00100000:0.0:1789760778.298534:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708408357803 time 708418242363 state 0x2 ref 0 00000100:00100000:3.0:1789760778.307889:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.308433:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785344 00000100:00100000:2.0:1789760778.308448:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785344 00000100:00100000:2.0:1789760778.308529:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708408357803 time 708418242363 state 0x0 ref 1 00000100:00100000:2.0:1789760778.308548:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708418502779 time 708418242363 state 0x1 ref 1 00000100:00100000:2.0:1789760778.308577:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708418502779 time 708433183152 state 0x0 ref 1 00000100:00100000:2.0:1789760778.308590:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503785344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.308674:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76705792, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24950 00010000:00010000:2.0:1789760778.308686:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.308711:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.308717:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.308725:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.308750:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.312674:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986034 is committed 00002000:00010000:2.0:1789760778.312858:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76705792 -> 76709888 00002000:00010000:2.0:1789760778.312863:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149840 -> 149848 00010000:00010000:2.0:1789760778.312880:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.312896:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.312906:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.312913:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.312919:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.312929:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.312954:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.312963:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f081a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76705792->76709887] (req 76705792->76709887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.313083:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503785344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4487us (5202us total) trans 4294986034 rc 0/0 00000100:00100000:2.0:1789760778.313095:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18738 00000100:00100000:2.0:1789760778.313108:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708418502779 time 708433183152 state 0x2 ref 0 00000100:00100000:1.0:1789760778.318982:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.319140:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785472 00000100:00100000:2.0:1789760778.319162:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785472 00000100:00100000:2.0:1789760778.319239:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708418502779 time 708433183152 state 0x0 ref 1 00000100:00100000:2.0:1789760778.319258:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708433443568 time 708433183152 state 0x1 ref 1 00000100:00100000:2.0:1789760778.319296:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708433443568 time 708443902394 state 0x0 ref 1 00000100:00100000:2.0:1789760778.319310:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.319722:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76709888, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24958 00010000:00010000:2.0:1789760778.319735:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.319756:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.319761:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.319769:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.319785:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.324836:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986035 is committed 00002000:00010000:2.0:1789760778.325000:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76709888 -> 76713984 00002000:00010000:2.0:1789760778.325007:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149848 -> 149856 00010000:00010000:2.0:1789760778.325023:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.325039:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.325049:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.325058:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.325064:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.325075:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.325102:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.325112:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0821 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76709888->76713983] (req 76709888->76713983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.325251:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5936us (6288us total) trans 4294986035 rc 0/0 00000100:00100000:2.0:1789760778.325265:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18739 00000100:00100000:2.0:1789760778.325279:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708433443568 time 708443902394 state 0x2 ref 0 00000100:00100000:3.0:1789760778.329559:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.329708:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785600 00000100:00100000:0.0:1789760778.329722:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785600 00000100:00100000:0.0:1789760778.329801:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708433443568 time 708443902394 state 0x0 ref 1 00000100:00100000:0.0:1789760778.329820:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708444162810 time 708443902394 state 0x1 ref 1 00000100:00100000:0.0:1789760778.329851:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708444162810 time 708454457328 state 0x0 ref 1 00000100:00100000:0.0:1789760778.329865:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.329926:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76713984, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24960 00010000:00010000:0.0:1789760778.329937:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.329956:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.329963:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.329971:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.329986:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.333968:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986036 is committed 00002000:00010000:0.0:1789760778.334112:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76713984 -> 76718080 00002000:00010000:0.0:1789760778.334119:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149856 -> 149864 00010000:00010000:0.0:1789760778.334138:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.334152:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.334161:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.334170:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.334176:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.334186:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.334212:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.334227:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0828 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76713984->76718079] (req 76713984->76718079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.334357:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4486us (4807us total) trans 4294986036 rc 0/0 00000100:00100000:0.0:1789760778.334370:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18740 00000100:00100000:0.0:1789760778.334384:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708444162810 time 708454457328 state 0x2 ref 0 00000100:00100000:3.0:1789760778.340574:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.340720:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785728 00000100:00100000:0.0:1789760778.340734:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785728 00000100:00100000:0.0:1789760778.340812:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708444162810 time 708454457328 state 0x0 ref 1 00000100:00100000:0.0:1789760778.340831:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708454717744 time 708454457328 state 0x1 ref 1 00000100:00100000:0.0:1789760778.340865:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708454717744 time 708465471308 state 0x0 ref 1 00000100:00100000:0.0:1789760778.340880:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.340947:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76718080, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24968 00010000:00010000:0.0:1789760778.340958:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.340978:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.340983:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.340990:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.341012:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.346695:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986037 is committed 00002000:00010000:0.0:1789760778.346823:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76718080 -> 76722176 00002000:00010000:0.0:1789760778.346828:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149864 -> 149872 00010000:00010000:0.0:1789760778.346842:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.346854:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.346862:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.346870:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.346874:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.346882:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.346899:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.346905:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f082f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76718080->76722175] (req 76718080->76722175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.347064:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6183us (6500us total) trans 4294986037 rc 0/0 00000100:00100000:0.0:1789760778.347076:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18741 00000100:00100000:0.0:1789760778.347085:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708454717744 time 708465471308 state 0x2 ref 0 00000100:00100000:1.0:1789760778.354333:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.355343:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785856 00000100:00100000:2.0:1789760778.355360:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785856 00000100:00100000:2.0:1789760778.355482:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708454717744 time 708465471308 state 0x0 ref 1 00000100:00100000:2.0:1789760778.355504:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708465731724 time 708465471308 state 0x1 ref 1 00000100:00100000:2.0:1789760778.355620:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708465731724 time 708480224394 state 0x0 ref 1 00000100:00100000:2.0:1789760778.355644:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.355723:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76722176, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24970 00010000:00010000:2.0:1789760778.355737:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.355758:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.355763:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.355772:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.355788:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.363980:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986038 is committed 00002000:00010000:2.0:1789760778.364206:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76722176 -> 76726272 00002000:00010000:2.0:1789760778.364212:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149872 -> 149880 00010000:00010000:2.0:1789760778.364226:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.364239:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.364248:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.364254:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.364258:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.364267:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.364288:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.364295:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0836 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76722176->76726271] (req 76722176->76726271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.364433:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8759us (10073us total) trans 4294986038 rc 0/0 00000100:00100000:2.0:1789760778.364443:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18742 00000100:00100000:2.0:1789760778.364452:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708465731724 time 708480224394 state 0x2 ref 0 00000100:00100000:3.0:1789760778.371649:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.371968:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503785984 00000100:00100000:2.0:1789760778.371979:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503785984 00000100:00100000:2.0:1789760778.372034:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708465731724 time 708480224394 state 0x0 ref 1 00000100:00100000:2.0:1789760778.372050:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708480484810 time 708480224394 state 0x1 ref 1 00000100:00100000:2.0:1789760778.372122:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708480484810 time 708496727488 state 0x0 ref 1 00000100:00100000:2.0:1789760778.372134:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.372179:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76726272, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24978 00010000:00010000:2.0:1789760778.372198:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.372213:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.372217:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.372223:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.372236:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.380456:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986039 is committed 00002000:00010000:2.0:1789760778.380567:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76726272 -> 76730368 00002000:00010000:2.0:1789760778.380573:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149880 -> 149888 00010000:00010000:2.0:1789760778.380603:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.380623:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.380732:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.380741:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.380746:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.380755:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.380775:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.380844:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f083d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76726272->76730367] (req 76726272->76730367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.380938:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503785984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8800us (9300us total) trans 4294986039 rc 0/0 00000100:00100000:2.0:1789760778.380947:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18743 00000100:00100000:2.0:1789760778.380957:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708480484810 time 708496727488 state 0x2 ref 0 00000100:00100000:3.0:1789760778.387767:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.388029:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503786112 00000100:00100000:0.0:1789760778.388039:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503786112 00000100:00100000:0.0:1789760778.388095:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708480484810 time 708496727488 state 0x0 ref 1 00000100:00100000:0.0:1789760778.388109:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708496987904 time 708496727488 state 0x1 ref 1 00000100:00100000:0.0:1789760778.388141:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708496987904 time 708512747353 state 0x0 ref 1 00000100:00100000:0.0:1789760778.388152:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.388194:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76730368, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24980 00010000:00010000:0.0:1789760778.388202:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.388218:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.388222:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.388227:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.388239:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.396307:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986040 is committed 00002000:00010000:0.0:1789760778.396947:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76730368 -> 76734464 00002000:00010000:0.0:1789760778.396954:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149888 -> 149896 00010000:00010000:0.0:1789760778.396968:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.396981:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.396991:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.396999:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.397005:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.397014:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.397036:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.397044:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0844 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76730368->76734463] (req 76730368->76734463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.397198:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9042us (9436us total) trans 4294986040 rc 0/0 00000100:00100000:0.0:1789760778.397209:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18744 00000100:00100000:0.0:1789760778.397219:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708496987904 time 708512747353 state 0x2 ref 0 00000100:00100000:1.0:1789760778.406857:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.406957:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503786240 00000100:00100000:0.0:1789760778.406967:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503786240 00000100:00100000:0.0:1789760778.407023:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708496987904 time 708512747353 state 0x0 ref 1 00000100:00100000:0.0:1789760778.407037:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708513007769 time 708512747353 state 0x1 ref 1 00000100:00100000:0.0:1789760778.407069:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708513007769 time 708531675768 state 0x0 ref 1 00000100:00100000:0.0:1789760778.407081:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.407128:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76734464, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24988 00010000:00010000:0.0:1789760778.407136:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.407152:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.407156:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.407163:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.407177:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.417071:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986041 is committed 00002000:00010000:0.0:1789760778.417488:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76734464 -> 76738560 00002000:00010000:0.0:1789760778.417493:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149896 -> 149904 00010000:00010000:0.0:1789760778.417507:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.417520:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.417528:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.417535:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.417540:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.417549:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.417570:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.417578:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f084b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76734464->76738559] (req 76734464->76738559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.417865:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10779us (11010us total) trans 4294986041 rc 0/0 00000100:00100000:0.0:1789760778.417876:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18745 00000100:00100000:0.0:1789760778.417886:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708513007769 time 708531675768 state 0x2 ref 0 00000100:00100000:3.0:1789760778.422280:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.422382:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503786368 00000100:00100000:2.0:1789760778.422421:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503786368 00000100:00100000:2.0:1789760778.422487:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708513007769 time 708531675768 state 0x0 ref 1 00000100:00100000:2.0:1789760778.422504:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708531936184 time 708531675768 state 0x1 ref 1 00000100:00100000:2.0:1789760778.422577:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708531936184 time 708547182232 state 0x0 ref 1 00000100:00100000:2.0:1789760778.422609:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.422660:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76738560, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24990 00010000:00010000:2.0:1789760778.422669:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.422684:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.422689:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.422694:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.422708:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.429094:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986042 is committed 00002000:00010000:2.0:1789760778.429325:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76738560 -> 76742656 00002000:00010000:2.0:1789760778.429332:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149904 -> 149912 00010000:00010000:2.0:1789760778.429344:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.429356:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.429370:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.429378:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.429383:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.429431:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.429450:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.429457:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0852 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76738560->76742655] (req 76738560->76742655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.429546:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6937us (7278us total) trans 4294986042 rc 0/0 00000100:00100000:2.0:1789760778.429556:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18746 00000100:00100000:2.0:1789760778.429565:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708531936184 time 708547182232 state 0x2 ref 0 00000100:00100000:3.0:1789760778.432781:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.432899:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503786496 00000100:00100000:2.0:1789760778.432908:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503786496 00000100:00100000:2.0:1789760778.432952:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708531936184 time 708547182232 state 0x0 ref 1 00000100:00100000:2.0:1789760778.432962:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708547442648 time 708547182232 state 0x1 ref 1 00000100:00100000:2.0:1789760778.432988:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708547442648 time 708557595473 state 0x0 ref 1 00000100:00100000:2.0:1789760778.432999:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.433046:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76742656, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24998 00010000:00010000:2.0:1789760778.433056:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433072:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.433076:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.433082:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.433096:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.439820:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986043 is committed 00002000:00010000:1.0:1789760778.440134:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76742656 -> 76746752 00002000:00010000:1.0:1789760778.440143:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149912 -> 149920 00010000:00010000:1.0:1789760778.440159:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.440173:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.440183:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.440212:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.440218:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.440230:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.440255:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.440264:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0859 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76742656->76746751] (req 76742656->76746751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.440547:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503786496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7544us (7769us total) trans 4294986043 rc 0/0 00000100:00100000:1.0:1789760778.440562:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18747 00000100:00100000:1.0:1789760778.440571:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708547442648 time 708557595473 state 0x2 ref 0 00000100:00100000:3.0:1789760778.454450:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.455348:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503786624 00000100:00100000:0.0:1789760778.455363:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503786624 00000100:00100000:0.0:1789760778.455492:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708547442648 time 708557595473 state 0x0 ref 1 00000100:00100000:0.0:1789760778.455513:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708557855889 time 708557595473 state 0x1 ref 1 00000100:00100000:0.0:1789760778.455541:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708557855889 time 708580146383 state 0x0 ref 1 00000100:00100000:0.0:1789760778.455560:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503786624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.455623:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76746752, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249a0 00010000:00010000:0.0:1789760778.455644:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.455660:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.455664:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.455671:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.455693:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.463320:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986044 is committed 00002000:00010000:0.0:1789760778.463665:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76746752 -> 76750848 00002000:00010000:0.0:1789760778.463671:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149920 -> 149928 00010000:00010000:0.0:1789760778.463683:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.463702:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.463711:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.463718:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.463723:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.463732:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.463753:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.463761:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0860 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76746752->76750847] (req 76746752->76750847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.463862:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503786624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8304us (9419us total) trans 4294986044 rc 0/0 00000100:00100000:0.0:1789760778.463874:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18748 00000100:00100000:0.0:1789760778.463884:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708557855889 time 708580146383 state 0x2 ref 0 00000100:00100000:1.0:1789760778.467883:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.468150:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503786752 00000100:00100000:0.0:1789760778.468162:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503786752 00000100:00100000:0.0:1789760778.468218:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708557855889 time 708580146383 state 0x0 ref 1 00000100:00100000:0.0:1789760778.468232:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708580406799 time 708580146383 state 0x1 ref 1 00000100:00100000:0.0:1789760778.468261:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708580406799 time 708592867968 state 0x0 ref 1 00000100:00100000:0.0:1789760778.468274:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503786752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.468329:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76750848, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249a8 00010000:00010000:0.0:1789760778.468339:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.468356:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.468361:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.468368:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.468383:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.476707:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986045 is committed 00002000:00010000:0.0:1789760778.476895:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76750848 -> 76754944 00002000:00010000:0.0:1789760778.476901:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149928 -> 149936 00010000:00010000:0.0:1789760778.476913:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476926:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476941:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476948:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.476954:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476963:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476983:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.476992:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0867 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76750848->76754943] (req 76750848->76754943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.477136:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503786752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8859us (9257us total) trans 4294986045 rc 0/0 00000100:00100000:0.0:1789760778.477146:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18749 00000100:00100000:0.0:1789760778.477156:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708580406799 time 708592867968 state 0x2 ref 0 00000100:00100000:3.0:1789760778.484526:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.484779:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503786880 00000100:00100000:2.0:1789760778.484789:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503786880 00000100:00100000:2.0:1789760778.484842:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708580406799 time 708592867968 state 0x0 ref 1 00000100:00100000:2.0:1789760778.484857:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708593128384 time 708592867968 state 0x1 ref 1 00000100:00100000:2.0:1789760778.485016:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708593128384 time 708609561223 state 0x0 ref 1 00000100:00100000:2.0:1789760778.485032:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503786880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.485081:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76754944, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249b0 00010000:00010000:2.0:1789760778.485090:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.485105:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.485110:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.485116:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.485129:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.494334:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986046 is committed 00002000:00010000:1.0:1789760778.494695:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76754944 -> 76759040 00002000:00010000:1.0:1789760778.494700:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149936 -> 149944 00010000:00010000:1.0:1789760778.494713:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.494724:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.494732:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.494739:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.494743:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.494751:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.494770:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.494782:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f086e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76754944->76759039] (req 76754944->76759039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.494878:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503786880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9846us (10368us total) trans 4294986046 rc 0/0 00000100:00100000:1.0:1789760778.494889:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18750 00000100:00100000:1.0:1789760778.494899:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708593128384 time 708609561223 state 0x2 ref 0 00000100:00100000:2.0:1789760778.498222:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.498366:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787008 00000100:00100000:1.0:1789760778.498375:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787008 00000100:00100000:1.0:1789760778.498467:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708593128384 time 708609561223 state 0x0 ref 1 00000100:00100000:1.0:1789760778.498482:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708609821639 time 708609561223 state 0x1 ref 1 00000100:00100000:1.0:1789760778.498502:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708609821639 time 708623109224 state 0x0 ref 1 00000100:00100000:1.0:1789760778.498514:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503787008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.498570:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76759040, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249b8 00010000:00010000:1.0:1789760778.498579:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.498615:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.498620:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.498627:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.498645:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.503858:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986047 is committed 00002000:00010000:1.0:1789760778.504044:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76759040 -> 76763136 00002000:00010000:1.0:1789760778.504049:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149944 -> 149952 00010000:00010000:1.0:1789760778.508247:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.508268:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.508278:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.508483:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.508488:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.508498:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.508532:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.508541:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0875 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76759040->76763135] (req 76759040->76763135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.508674:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503787008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10156us (10457us total) trans 4294986047 rc 0/0 00000100:00100000:1.0:1789760778.508684:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18751 00000100:00100000:1.0:1789760778.508693:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708609821639 time 708623109224 state 0x2 ref 0 00000100:00100000:0.0:1789760778.517854:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.519912:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787136 00000100:00100000:2.0:1789760778.519929:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787136 00000100:00100000:2.0:1789760778.519989:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708609821639 time 708623109224 state 0x0 ref 1 00000100:00100000:2.0:1789760778.520004:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708623369640 time 708623109224 state 0x1 ref 1 00000100:00100000:2.0:1789760778.520166:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708623369640 time 708644770911 state 0x0 ref 1 00000100:00100000:2.0:1789760778.520181:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503787136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.520232:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76763136, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249c0 00010000:00010000:2.0:1789760778.520241:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.520257:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.520262:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.520268:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.520282:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.527547:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986048 is committed 00002000:00010000:2.0:1789760778.527751:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76763136 -> 76767232 00002000:00010000:2.0:1789760778.527757:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149952 -> 149960 00010000:00010000:2.0:1789760778.527769:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.527782:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.527791:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.527798:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.527803:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.527811:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.527835:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.527842:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f087c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76763136->76767231] (req 76763136->76767231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.527928:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503787136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7746us (10080us total) trans 4294986048 rc 0/0 00000100:00100000:2.0:1789760778.527937:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18752 00000100:00100000:2.0:1789760778.527947:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708623369640 time 708644770911 state 0x2 ref 0 00000100:00100000:3.0:1789760778.534201:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.534811:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787264 00000100:00100000:0.0:1789760778.534820:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787264 00000100:00100000:0.0:1789760778.534873:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708623369640 time 708644770911 state 0x0 ref 1 00000100:00100000:0.0:1789760778.534888:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708645031327 time 708644770911 state 0x1 ref 1 00000100:00100000:0.0:1789760778.534962:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708645031327 time 708659568323 state 0x0 ref 1 00000100:00100000:0.0:1789760778.534975:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503787264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.535023:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76767232, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249c8 00010000:00010000:0.0:1789760778.535033:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.535049:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.535054:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.535061:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.535074:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.540050:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986049 is committed 00002000:00010000:0.0:1789760778.540166:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76767232 -> 76771328 00002000:00010000:0.0:1789760778.540171:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149960 -> 149968 00010000:00010000:0.0:1789760778.540184:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.540198:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.540212:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.540218:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.540223:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.540232:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.540251:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.540259:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0883 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76767232->76771327] (req 76767232->76771327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.540531:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503787264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5551us (6333us total) trans 4294986049 rc 0/0 00000100:00100000:0.0:1789760778.540542:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18753 00000100:00100000:0.0:1789760778.540552:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708645031327 time 708659568323 state 0x2 ref 0 00000100:00100000:0.0:1789760778.544279:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.547692:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787392 00000100:00100000:1.0:1789760778.547708:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787392 00000100:00100000:1.0:1789760778.547779:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708645031327 time 708659568323 state 0x0 ref 1 00000100:00100000:1.0:1789760778.547796:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708659828739 time 708659568323 state 0x1 ref 1 00000100:00100000:1.0:1789760778.547834:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708659828739 time 708672440571 state 0x0 ref 1 00000100:00100000:1.0:1789760778.547850:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.547912:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76771328, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249d0 00010000:00010000:1.0:1789760778.547922:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.547938:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.547943:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.547949:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.547963:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.553983:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986050 is committed 00002000:00010000:1.0:1789760778.554147:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76771328 -> 76775424 00002000:00010000:1.0:1789760778.554152:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149968 -> 149976 00010000:00010000:1.0:1789760778.554165:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.554177:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.554185:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.554192:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.554197:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.554207:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.554227:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.554234:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f088a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76771328->76775423] (req 76771328->76775423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.554386:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6530us (10107us total) trans 4294986050 rc 0/0 00000100:00100000:1.0:1789760778.554548:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18754 00000100:00100000:1.0:1789760778.554559:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708659828739 time 708672440571 state 0x2 ref 0 00000100:00100000:1.0:1789760778.559943:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.560146:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787520 00000100:00100000:2.0:1789760778.560155:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787520 00000100:00100000:2.0:1789760778.560208:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708659828739 time 708672440571 state 0x0 ref 1 00000100:00100000:2.0:1789760778.560224:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708672700987 time 708672440571 state 0x1 ref 1 00000100:00100000:2.0:1789760778.560245:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708672700987 time 708684852094 state 0x0 ref 1 00000100:00100000:2.0:1789760778.560257:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503787520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.560302:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76775424, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249d8 00010000:00010000:2.0:1789760778.560311:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.560326:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.560331:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.560338:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.560351:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.565442:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986051 is committed 00002000:00010000:2.0:1789760778.565583:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76775424 -> 76779520 00002000:00010000:2.0:1789760778.565588:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149976 -> 149984 00010000:00010000:2.0:1789760778.565614:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.565627:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.565636:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.565642:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.565647:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.565656:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.565680:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.565693:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0891 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76775424->76779519] (req 76775424->76779519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.565830:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503787520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5570us (5894us total) trans 4294986051 rc 0/0 00000100:00100000:2.0:1789760778.565840:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18755 00000100:00100000:2.0:1789760778.565849:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708672700987 time 708684852094 state 0x2 ref 0 00000100:00100000:3.0:1789760778.569036:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.569811:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787648 00000100:00100000:2.0:1789760778.569823:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787648 00000100:00100000:2.0:1789760778.569882:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708672700987 time 708684852094 state 0x0 ref 1 00000100:00100000:2.0:1789760778.569897:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708685112510 time 708684852094 state 0x1 ref 1 00000100:00100000:2.0:1789760778.570035:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708685112510 time 708694640421 state 0x0 ref 1 00000100:00100000:2.0:1789760778.570049:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.570096:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76779520, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249e0 00010000:00010000:2.0:1789760778.570105:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.570120:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.570124:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.570130:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.570147:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.576214:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986052 is committed 00002000:00010000:2.0:1789760778.576328:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76779520 -> 76783616 00002000:00010000:2.0:1789760778.576333:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149984 -> 149992 00010000:00010000:2.0:1789760778.576347:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.576361:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.576370:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.576377:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.576382:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.576420:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.576442:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.576449:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0898 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76779520->76783615] (req 76779520->76783615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.576537:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6486us (7505us total) trans 4294986052 rc 0/0 00000100:00100000:2.0:1789760778.576548:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18756 00000100:00100000:2.0:1789760778.576557:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708685112510 time 708694640421 state 0x2 ref 0 00000100:00100000:0.0:1789760778.579573:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.579743:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787776 00000100:00100000:2.0:1789760778.579753:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787776 00000100:00100000:2.0:1789760778.579805:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708685112510 time 708694640421 state 0x0 ref 1 00000100:00100000:2.0:1789760778.579818:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708694900837 time 708694640421 state 0x1 ref 1 00000100:00100000:2.0:1789760778.579855:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708694900837 time 708704461723 state 0x0 ref 1 00000100:00100000:2.0:1789760778.579868:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.579924:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76783616, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249e8 00010000:00010000:2.0:1789760778.579934:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.579949:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.579953:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.579960:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.579974:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.589443:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986053 is committed 00002000:00010000:0.0:1789760778.589569:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76783616 -> 76787712 00002000:00010000:0.0:1789760778.589576:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 149992 -> 150000 00010000:00010000:0.0:1789760778.589599:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.589613:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.589622:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.589629:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.589634:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.589643:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.589663:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.589670:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f089f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76783616->76787711] (req 76783616->76787711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.589769:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9897us (10200us total) trans 4294986053 rc 0/0 00000100:00100000:0.0:1789760778.589779:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18757 00000100:00100000:0.0:1789760778.589789:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708694900837 time 708704461723 state 0x2 ref 0 00000100:00100000:0.0:1789760778.593801:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.593908:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503787904 00000100:00100000:1.0:1789760778.593918:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503787904 00000100:00100000:1.0:1789760778.593972:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708694900837 time 708704461723 state 0x0 ref 1 00000100:00100000:1.0:1789760778.593988:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708704722139 time 708704461723 state 0x1 ref 1 00000100:00100000:1.0:1789760778.594051:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708704722139 time 708718656716 state 0x0 ref 1 00000100:00100000:1.0:1789760778.594064:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.594109:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76787712, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249f0 00010000:00010000:1.0:1789760778.594119:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.594134:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.594138:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.594145:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.594159:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.601042:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986054 is committed 00002000:00010000:1.0:1789760778.602586:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76787712 -> 76791808 00002000:00010000:1.0:1789760778.602609:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150000 -> 150008 00010000:00010000:1.0:1789760778.602624:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602637:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602646:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602654:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.602659:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602668:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602802:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602809:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76787712->76791807] (req 76787712->76791807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.602929:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503787904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8864us (9136us total) trans 4294986054 rc 0/0 00000100:00100000:1.0:1789760778.602939:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18758 00000100:00100000:1.0:1789760778.602950:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708704722139 time 708718656716 state 0x2 ref 0 00000100:00100000:3.0:1789760778.614290:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.614613:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788032 00000100:00100000:1.0:1789760778.614622:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788032 00000100:00100000:1.0:1789760778.614688:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708704722139 time 708718656716 state 0x0 ref 1 00000100:00100000:1.0:1789760778.614703:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708718917132 time 708718656716 state 0x1 ref 1 00000100:00100000:1.0:1789760778.614729:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708718917132 time 708739336457 state 0x0 ref 1 00000100:00100000:1.0:1789760778.614743:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503788032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.614794:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76791808, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x249f8 00010000:00010000:1.0:1789760778.614803:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.614817:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.614821:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.614828:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.614840:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.620220:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986055 is committed 00002000:00010000:1.0:1789760778.620805:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76791808 -> 76795904 00002000:00010000:1.0:1789760778.620811:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150008 -> 150016 00010000:00010000:1.0:1789760778.620824:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.620842:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.620850:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.620857:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.620862:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.620872:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.620891:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.620899:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76791808->76795903] (req 76791808->76795903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.621004:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503788032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6249us (6710us total) trans 4294986055 rc 0/0 00000100:00100000:1.0:1789760778.621015:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18759 00000100:00100000:1.0:1789760778.621025:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708718917132 time 708739336457 state 0x2 ref 0 00000100:00100000:0.0:1789760778.628585:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.628718:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788160 00000100:00100000:2.0:1789760778.628728:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788160 00000100:00100000:2.0:1789760778.628781:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708718917132 time 708739336457 state 0x0 ref 1 00000100:00100000:2.0:1789760778.628797:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708739596873 time 708739336457 state 0x1 ref 1 00000100:00100000:2.0:1789760778.628828:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708739596873 time 708753435128 state 0x0 ref 1 00000100:00100000:2.0:1789760778.628840:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.628888:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76795904, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a00 00010000:00010000:2.0:1789760778.628897:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.628914:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.628919:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.628925:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.628939:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.635627:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986056 is committed 00002000:00010000:2.0:1789760778.635813:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76795904 -> 76800000 00002000:00010000:2.0:1789760778.635818:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150016 -> 150024 00010000:00010000:2.0:1789760778.635831:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.635851:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.635861:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.635868:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.635873:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.635882:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.635979:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.635989:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76795904->76799999] (req 76795904->76799999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.636164:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7322us (7584us total) trans 4294986056 rc 0/0 00000100:00100000:2.0:1789760778.636175:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18760 00000100:00100000:2.0:1789760778.636185:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708739596873 time 708753435128 state 0x2 ref 0 00000100:00100000:0.0:1789760778.642817:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760778.642906:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788288 00000100:00100000:3.0:1789760778.642917:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788288 00000100:00100000:3.0:1789760778.642976:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708739596873 time 708753435128 state 0x0 ref 1 00000100:00100000:3.0:1789760778.642992:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708753695544 time 708753435128 state 0x1 ref 1 00000100:00100000:3.0:1789760778.643028:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708753695544 time 708767635039 state 0x0 ref 1 00000100:00100000:3.0:1789760778.643042:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760778.643094:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76800000, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a08 00010000:00010000:3.0:1789760778.643103:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.643119:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760778.643123:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.643130:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.643144:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.648347:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986057 is committed 00002000:00010000:3.0:1789760778.648459:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76800000 -> 76804096 00002000:00010000:3.0:1789760778.648465:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150024 -> 150032 00010000:00010000:3.0:1789760778.648477:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.648494:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.648503:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.648510:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760778.648515:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.648524:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.648543:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.648551:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f08bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76800000->76804095] (req 76800000->76804095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.648653:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5608us (5841us total) trans 4294986057 rc 0/0 00000100:00100000:3.0:1789760778.648663:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18761 00000100:00100000:3.0:1789760778.648673:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708753695544 time 708767635039 state 0x2 ref 0 00000100:00100000:3.0:1789760778.652562:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.653800:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788416 00000100:00100000:1.0:1789760778.653814:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788416 00000100:00100000:1.0:1789760778.653878:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708753695544 time 708767635039 state 0x0 ref 1 00000100:00100000:1.0:1789760778.653894:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708767895455 time 708767635039 state 0x1 ref 1 00000100:00100000:1.0:1789760778.653930:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708767895455 time 708778536805 state 0x0 ref 1 00000100:00100000:1.0:1789760778.653943:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc15450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503788416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.653996:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76804096, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a10 00010000:00010000:1.0:1789760778.654005:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.654022:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.654026:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.654033:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.654048:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.659695:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986058 is committed 00002000:00010000:1.0:1789760778.659885:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76804096 -> 76808192 00002000:00010000:1.0:1789760778.659890:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150032 -> 150040 00010000:00010000:1.0:1789760778.659903:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.659917:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.659931:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.659939:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.659944:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.659953:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.659973:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.659981:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76804096->76808191] (req 76804096->76808191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.660090:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc15450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503788416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6144us (7779us total) trans 4294986058 rc 0/0 00000100:00100000:1.0:1789760778.660100:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18762 00000100:00100000:1.0:1789760778.660110:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708767895455 time 708778536805 state 0x2 ref 0 00000100:00100000:0.0:1789760778.667013:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.667104:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788544 00000100:00100000:1.0:1789760778.667115:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788544 00000100:00100000:1.0:1789760778.667168:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708767895455 time 708778536805 state 0x0 ref 1 00000100:00100000:1.0:1789760778.667188:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708778797221 time 708778536805 state 0x1 ref 1 00000100:00100000:1.0:1789760778.667219:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708778797221 time 708791826339 state 0x0 ref 1 00000100:00100000:1.0:1789760778.667231:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.667275:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76808192, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a18 00010000:00010000:1.0:1789760778.667285:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.667302:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.667306:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.667312:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.667573:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.676677:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986059 is committed 00002000:00010000:1.0:1789760778.676856:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76808192 -> 76812288 00002000:00010000:1.0:1789760778.676862:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150040 -> 150048 00010000:00010000:1.0:1789760778.676876:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.676900:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.676909:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.676923:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.676928:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.676937:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.676957:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.676965:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f08c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76808192->76812287] (req 76808192->76812287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.677068:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9834us (10060us total) trans 4294986059 rc 0/0 00000100:00100000:1.0:1789760778.677078:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18763 00000100:00100000:1.0:1789760778.677087:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708778797221 time 708791826339 state 0x2 ref 0 00000100:00100000:0.0:1789760778.683958:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.684057:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788672 00000100:00100000:2.0:1789760778.684071:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788672 00000100:00100000:2.0:1789760778.684136:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708778797221 time 708791826339 state 0x0 ref 1 00000100:00100000:2.0:1789760778.684154:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708792086755 time 708791826339 state 0x1 ref 1 00000100:00100000:2.0:1789760778.684203:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708792086755 time 708808807684 state 0x0 ref 1 00000100:00100000:2.0:1789760778.684221:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.684698:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76812288, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a20 00010000:00010000:2.0:1789760778.684709:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.684727:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.684731:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.684737:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.684751:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.690973:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986060 is committed 00002000:00010000:2.0:1789760778.691130:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76812288 -> 76816384 00002000:00010000:2.0:1789760778.691135:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150048 -> 150056 00010000:00010000:2.0:1789760778.691148:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.691161:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.691174:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.691181:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.691185:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.691193:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.691212:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.691220:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76812288->76816383] (req 76812288->76816383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.691915:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7690us (7958us total) trans 4294986060 rc 0/0 00000100:00100000:2.0:1789760778.691930:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18764 00000100:00100000:2.0:1789760778.691942:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708792086755 time 708808807684 state 0x2 ref 0 00000100:00100000:3.0:1789760778.702257:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.705450:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788800 00000100:00100000:0.0:1789760778.705465:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788800 00000100:00100000:0.0:1789760778.705721:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708792086755 time 708808807684 state 0x0 ref 1 00000100:00100000:0.0:1789760778.705736:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708809068100 time 708808807684 state 0x1 ref 1 00000100:00100000:0.0:1789760778.705821:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708809068100 time 708830426563 state 0x0 ref 1 00000100:00100000:0.0:1789760778.705836:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503788800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.705886:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76816384, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a28 00010000:00010000:0.0:1789760778.705904:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.705920:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.705924:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.705932:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.705949:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.711957:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986061 is committed 00002000:00010000:3.0:1789760778.712195:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76816384 -> 76820480 00002000:00010000:3.0:1789760778.712201:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150056 -> 150064 00010000:00010000:3.0:1789760778.712214:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.712226:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.712236:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.712244:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760778.712249:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.712259:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.712279:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.712288:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f08d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76816384->76820479] (req 76816384->76820479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.713173:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503788800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7333us (10918us total) trans 4294986061 rc 0/0 00000100:00100000:3.0:1789760778.713187:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18765 00000100:00100000:3.0:1789760778.713197:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708809068100 time 708830426563 state 0x2 ref 0 00000100:00100000:3.0:1789760778.726514:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.726654:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503788928 00000100:00100000:1.0:1789760778.726666:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503788928 00000100:00100000:1.0:1789760778.726722:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708809068100 time 708830426563 state 0x0 ref 1 00000100:00100000:1.0:1789760778.726738:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708830686979 time 708830426563 state 0x1 ref 1 00000100:00100000:1.0:1789760778.726778:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708830686979 time 708851382604 state 0x0 ref 1 00000100:00100000:1.0:1789760778.726798:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.726865:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76820480, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a30 00010000:00010000:1.0:1789760778.726874:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.726889:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.726893:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.726900:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.726914:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.735474:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986062 is committed 00002000:00010000:1.0:1789760778.735674:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76820480 -> 76824576 00002000:00010000:1.0:1789760778.735680:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150064 -> 150072 00010000:00010000:1.0:1789760778.735693:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.735706:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.735714:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.735721:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.735726:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.735736:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.735755:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.735763:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f08de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76820480->76824575] (req 76820480->76824575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.735923:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503788928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9123us (9415us total) trans 4294986062 rc 0/0 00000100:00100000:1.0:1789760778.735935:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18766 00000100:00100000:1.0:1789760778.735945:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708830686979 time 708851382604 state 0x2 ref 0 00000100:00100000:1.0:1789760778.747542:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.747749:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789056 00000100:00100000:2.0:1789760778.747760:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789056 00000100:00100000:2.0:1789760778.747822:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708830686979 time 708851382604 state 0x0 ref 1 00000100:00100000:2.0:1789760778.747840:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708851643020 time 708851382604 state 0x1 ref 1 00000100:00100000:2.0:1789760778.747873:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708851643020 time 708872480178 state 0x0 ref 1 00000100:00100000:2.0:1789760778.747885:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.747934:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76824576, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a38 00010000:00010000:2.0:1789760778.747943:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.747959:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.747963:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.747969:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.747982:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.753458:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986063 is committed 00002000:00010000:2.0:1789760778.753647:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76824576 -> 76828672 00002000:00010000:2.0:1789760778.753652:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150072 -> 150080 00010000:00010000:2.0:1789760778.753665:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.753677:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.753686:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.753698:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.753703:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.753711:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.753730:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.753738:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76824576->76828671] (req 76824576->76828671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.753876:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5988us (6339us total) trans 4294986063 rc 0/0 00000100:00100000:2.0:1789760778.753886:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18767 00000100:00100000:2.0:1789760778.753896:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708851643020 time 708872480178 state 0x2 ref 0 00010000:00010000:2.0:1789760778.770572:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760778.772363:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.775158:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789184 00000100:00100000:2.0:1789760778.775173:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789184 00000100:00100000:2.0:1789760778.775236:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708851643020 time 708872480178 state 0x0 ref 1 00000100:00100000:2.0:1789760778.775252:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708872740594 time 708872480178 state 0x1 ref 1 00000100:00100000:2.0:1789760778.775474:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708872740594 time 708900079014 state 0x0 ref 1 00000100:00100000:2.0:1789760778.775500:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.775552:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76828672, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a40 00010000:00010000:2.0:1789760778.775561:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.775578:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.775583:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.775601:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.775615:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.782987:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986064 is committed 00002000:00010000:3.0:1789760778.783627:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76828672 -> 76832768 00002000:00010000:3.0:1789760778.783634:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150080 -> 150088 00010000:00010000:3.0:1789760778.783648:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.783660:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.783669:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.783677:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760778.783682:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.783691:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.783710:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.783723:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f08ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76828672->76832767] (req 76828672->76832767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.783870:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8379us (11512us total) trans 4294986064 rc 0/0 00000100:00100000:3.0:1789760778.783882:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18768 00000100:00100000:3.0:1789760778.783892:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708872740594 time 708900079014 state 0x2 ref 0 00000100:00100000:3.0:1789760778.790192:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.791003:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789312 00000100:00100000:0.0:1789760778.791014:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789312 00000100:00100000:0.0:1789760778.791072:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708872740594 time 708900079014 state 0x0 ref 1 00000100:00100000:0.0:1789760778.791087:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708900339430 time 708900079014 state 0x1 ref 1 00000100:00100000:0.0:1789760778.791110:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708900339430 time 708915716721 state 0x0 ref 1 00000100:00100000:0.0:1789760778.791122:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.791171:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76832768, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x24a48 00010000:00010000:0.0:1789760778.791180:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.791476:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760778.791482:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.791559:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.791575:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.801496:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986065 is committed 00002000:00010000:0.0:1789760778.801706:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76832768 -> 76836864 00002000:00010000:0.0:1789760778.801712:0:8434:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:0.0:1789760778.801714:0:8434:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00002000:00010000:0.0:1789760778.801716:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150088 -> 150096 00010000:00010000:0.0:1789760778.801729:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.801742:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.801750:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.801758:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.801763:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.801778:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.801798:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.801806:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f08f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76832768->76836863] (req 76832768->76836863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.801909:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10783us (11721us total) trans 4294986065 rc 0/0 00000100:00100000:0.0:1789760778.801919:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18769 00000100:00100000:0.0:1789760778.801945:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708900339430 time 708915716721 state 0x2 ref 0 00000100:00100000:1.0:1789760778.811909:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.812002:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789440 00000100:00100000:2.0:1789760778.812012:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789440 00000100:00100000:2.0:1789760778.816810:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708900339430 time 708915716721 state 0x0 ref 1 00000100:00100000:2.0:1789760778.817021:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708915977137 time 708915716721 state 0x1 ref 1 00000100:00100000:2.0:1789760778.817063:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708915977137 time 708941669513 state 0x0 ref 1 00000100:00100000:2.0:1789760778.817082:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503789440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.817132:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76836864, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a50 00010000:00010000:2.0:1789760778.817142:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.817159:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.817164:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.817170:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.817185:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.827077:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986066 is committed 00002000:00010000:2.0:1789760778.827784:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76836864 -> 76840960 00002000:00010000:2.0:1789760778.827791:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150096 -> 150104 00010000:00010000:2.0:1789760778.827806:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.827827:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.827841:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.827849:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.827854:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.827872:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.827897:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.827905:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f08fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76836864->76840959] (req 76836864->76840959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.828065:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503789440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10986us (16160us total) trans 4294986066 rc 0/0 00000100:00100000:2.0:1789760778.828076:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18770 00000100:00100000:2.0:1789760778.828084:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708915977137 time 708941669513 state 0x2 ref 0 00000100:00100000:3.0:1789760778.840251:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.840367:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789568 00000100:00100000:1.0:1789760778.840377:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789568 00000100:00100000:1.0:1789760778.840474:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708915977137 time 708941669513 state 0x0 ref 1 00000100:00100000:1.0:1789760778.840490:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708941929929 time 708941669513 state 0x1 ref 1 00000100:00100000:1.0:1789760778.840511:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708941929929 time 708965118072 state 0x0 ref 1 00000100:00100000:1.0:1789760778.840524:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503789568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.840573:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76840960, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a58 00010000:00010000:1.0:1789760778.840604:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.840621:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760778.840625:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.840632:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.840773:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.852408:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986067 is committed 00002000:00010000:1.0:1789760778.853169:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76840960 -> 76845056 00002000:00010000:1.0:1789760778.853176:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150104 -> 150112 00010000:00010000:1.0:1789760778.853190:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.853203:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.853213:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.853221:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.853226:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.853241:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.853263:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.853271:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0901 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76840960->76845055] (req 76840960->76845055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.853666:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503789568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13138us (13419us total) trans 4294986067 rc 0/0 00000100:00100000:1.0:1789760778.853677:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18771 00000100:00100000:1.0:1789760778.853687:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708941929929 time 708965118072 state 0x2 ref 0 00000100:00100000:3.0:1789760778.861825:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.862054:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789696 00000100:00100000:2.0:1789760778.862065:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789696 00000100:00100000:2.0:1789760778.862121:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708941929929 time 708965118072 state 0x0 ref 1 00000100:00100000:2.0:1789760778.862136:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708965378488 time 708965118072 state 0x1 ref 1 00000100:00100000:2.0:1789760778.862170:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708965378488 time 708986776802 state 0x0 ref 1 00000100:00100000:2.0:1789760778.862182:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.862235:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76845056, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a60 00010000:00010000:2.0:1789760778.862245:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.862274:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760778.862279:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.862286:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.862300:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.871086:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986068 is committed 00002000:00010000:0.0:1789760778.874892:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76845056 -> 76849152 00002000:00010000:0.0:1789760778.874900:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150112 -> 150120 00010000:00010000:0.0:1789760778.874915:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.874927:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.874937:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.874945:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.874949:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.874959:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.874978:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.874991:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0908 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76845056->76849151] (req 76845056->76849151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.875096:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503789696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12911us (13277us total) trans 4294986068 rc 0/0 00000100:00100000:0.0:1789760778.875106:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18772 00000100:00100000:0.0:1789760778.875116:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708965378488 time 708986776802 state 0x2 ref 0 00000100:00100000:2.0:1789760778.883302:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.883951:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789824 00000100:00100000:0.0:1789760778.883962:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789824 00000100:00100000:0.0:1789760778.884018:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708965378488 time 708986776802 state 0x0 ref 1 00000100:00100000:0.0:1789760778.884035:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708987037218 time 708986776802 state 0x1 ref 1 00000100:00100000:0.0:1789760778.884117:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708987037218 time 709008722637 state 0x0 ref 1 00000100:00100000:0.0:1789760778.884129:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503789824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.884175:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76849152, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a68 00010000:00010000:0.0:1789760778.884184:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.884200:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760778.884205:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.884211:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.884233:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.892264:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986069 is committed 00002000:00010000:3.0:1789760778.892649:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76849152 -> 76853248 00002000:00010000:3.0:1789760778.892655:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150120 -> 150128 00010000:00010000:3.0:1789760778.892669:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.892682:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.892691:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.892699:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760778.892705:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.892714:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.892734:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.892742:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f090f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76849152->76853247] (req 76849152->76853247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.892841:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503789824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8709us (9543us total) trans 4294986069 rc 0/0 00000100:00100000:3.0:1789760778.892852:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18773 00000100:00100000:3.0:1789760778.892862:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708987037218 time 709008722637 state 0x2 ref 0 00000100:00100000:3.0:1789760778.897137:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.897232:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503789952 00000100:00100000:1.0:1789760778.897242:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503789952 00000100:00100000:1.0:1789760778.897299:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 708987037218 time 709008722637 state 0x0 ref 1 00000100:00100000:1.0:1789760778.897314:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709008983053 time 709008722637 state 0x1 ref 1 00000100:00100000:1.0:1789760778.897346:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709008983053 time 709021953000 state 0x0 ref 1 00000100:00100000:1.0:1789760778.897358:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503789952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.897448:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76853248, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a70 00010000:00010000:1.0:1789760778.897456:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.897471:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760778.897476:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.897482:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.897496:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.904275:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986070 is committed 00002000:00010000:1.0:1789760778.904426:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76853248 -> 76857344 00002000:00010000:1.0:1789760778.904431:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150128 -> 150136 00010000:00010000:1.0:1789760778.904446:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.904460:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.904469:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.904476:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.904481:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.904491:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.904519:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.904528:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0916 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76853248->76857343] (req 76853248->76857343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.904756:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503789952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7393us (7624us total) trans 4294986070 rc 0/0 00000100:00100000:1.0:1789760778.904767:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18774 00000100:00100000:1.0:1789760778.904776:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709008983053 time 709021953000 state 0x2 ref 0 00000100:00100000:0.0:1789760778.907853:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.907974:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790080 00000100:00100000:1.0:1789760778.907982:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790080 00000100:00100000:1.0:1789760778.908034:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709008983053 time 709021953000 state 0x0 ref 1 00000100:00100000:1.0:1789760778.908048:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709022213416 time 709021953000 state 0x1 ref 1 00000100:00100000:1.0:1789760778.908090:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709022213416 time 709032696080 state 0x0 ref 1 00000100:00100000:1.0:1789760778.908102:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.908151:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76857344, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a78 00010000:00010000:1.0:1789760778.908160:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.908175:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760778.908180:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.908186:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.908200:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.914896:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986071 is committed 00002000:00010000:1.0:1789760778.916308:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76857344 -> 76861440 00002000:00010000:1.0:1789760778.916317:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150136 -> 150144 00010000:00010000:1.0:1789760778.916330:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.916343:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.916352:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.916360:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.916365:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.916375:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.916462:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.916477:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f091d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76857344->76861439] (req 76857344->76861439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.916646:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8540us (8796us total) trans 4294986071 rc 0/0 00000100:00100000:1.0:1789760778.916656:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18775 00000100:00100000:1.0:1789760778.916667:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709022213416 time 709032696080 state 0x2 ref 0 00000100:00100000:2.0:1789760778.920294:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.920465:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790208 00000100:00100000:0.0:1789760778.920480:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790208 00000100:00100000:0.0:1789760778.920538:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709022213416 time 709032696080 state 0x0 ref 1 00000100:00100000:0.0:1789760778.920552:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709032956496 time 709032696080 state 0x1 ref 1 00000100:00100000:0.0:1789760778.920685:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709032956496 time 709045291140 state 0x0 ref 1 00000100:00100000:0.0:1789760778.920698:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.920746:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76861440, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a80 00010000:00010000:0.0:1789760778.920755:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.920770:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760778.920775:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.920781:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.920794:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.927131:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986072 is committed 00002000:00010000:2.0:1789760778.927469:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76861440 -> 76865536 00002000:00010000:2.0:1789760778.927474:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150144 -> 150152 00010000:00010000:2.0:1789760778.927488:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.927500:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.927510:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.927768:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.927773:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.927784:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.927805:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.927814:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0924 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76861440->76865535] (req 76861440->76865535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.927946:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7244us (7656us total) trans 4294986072 rc 0/0 00000100:00100000:2.0:1789760778.927957:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18776 00000100:00100000:2.0:1789760778.927967:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709032956496 time 709045291140 state 0x2 ref 0 00000100:00100000:3.0:1789760778.933257:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.933357:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790336 00000100:00100000:0.0:1789760778.933368:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790336 00000100:00100000:0.0:1789760778.933662:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709032956496 time 709045291140 state 0x0 ref 1 00000100:00100000:0.0:1789760778.933677:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709045551556 time 709045291140 state 0x1 ref 1 00000100:00100000:0.0:1789760778.933701:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709045551556 time 709058307468 state 0x0 ref 1 00000100:00100000:0.0:1789760778.933714:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.933765:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76865536, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a88 00010000:00010000:0.0:1789760778.933774:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.933790:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760778.933795:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.933802:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.933984:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.941781:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986073 is committed 00002000:00010000:1.0:1789760778.942096:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76865536 -> 76869632 00002000:00010000:1.0:1789760778.942101:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150152 -> 150160 00010000:00010000:1.0:1789760778.942114:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.942125:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.942133:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.942140:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.942144:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.942154:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.942178:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.942185:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f092b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76865536->76869631] (req 76865536->76869631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.942279:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8562us (9027us total) trans 4294986073 rc 0/0 00000100:00100000:1.0:1789760778.942289:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18777 00000100:00100000:1.0:1789760778.942298:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709045551556 time 709058307468 state 0x2 ref 0 00000100:00100000:2.0:1789760778.948051:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.948217:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790464 00000100:00100000:1.0:1789760778.948228:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790464 00000100:00100000:1.0:1789760778.948285:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709045551556 time 709058307468 state 0x0 ref 1 00000100:00100000:1.0:1789760778.948300:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709058567884 time 709058307468 state 0x1 ref 1 00000100:00100000:1.0:1789760778.948447:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709058567884 time 709073051389 state 0x0 ref 1 00000100:00100000:1.0:1789760778.948461:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.948527:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76869632, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a90 00010000:00010000:1.0:1789760778.948537:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.948554:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760778.948559:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.948566:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.948579:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.953234:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986074 is committed 00002000:00010000:1.0:1789760778.953359:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76869632 -> 76873728 00002000:00010000:1.0:1789760778.953364:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150160 -> 150168 00010000:00010000:1.0:1789760778.953378:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.953423:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.953433:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.953441:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760778.953446:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.953455:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.953476:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.953491:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0932 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76869632->76873727] (req 76869632->76873727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.953599:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5128us (5544us total) trans 4294986074 rc 0/0 00000100:00100000:1.0:1789760778.953611:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18778 00000100:00100000:1.0:1789760778.953621:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709058567884 time 709073051389 state 0x2 ref 0 00000100:00100000:2.0:1789760778.956694:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760778.956818:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790592 00000100:00100000:1.0:1789760778.956827:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790592 00000100:00100000:1.0:1789760778.956883:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709058567884 time 709073051389 state 0x0 ref 1 00000100:00100000:1.0:1789760778.956899:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709073311805 time 709073051389 state 0x1 ref 1 00000100:00100000:1.0:1789760778.956930:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709073311805 time 709081536873 state 0x0 ref 1 00000100:00100000:1.0:1789760778.956943:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760778.956989:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76873728, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24a98 00010000:00010000:1.0:1789760778.956998:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.957013:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760778.957018:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.957025:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.957044:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.962943:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cf9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:afc717a1-11ba-4ef0-ab10-103aa692ca98:3637:1876699523608320:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963008:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963057:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523608832:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963087:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963107:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523609344:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963129:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963148:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523609856:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963168:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963193:0:3637:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1789760778.963200:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cf9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3637:1876699523608704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963229:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963249:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523609216:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963269:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963286:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523609728:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963306:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963338:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cf8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3638:1876699523608448:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963361:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963376:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3638:1876699523608960:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963413:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963431:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3638:1876699523609472:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963454:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963470:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523609984:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760778.963504:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963840:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1789760778.963847:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cfbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523608576:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963872:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963900:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3638:1876699523609088:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963921:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963939:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3638:1876699523609600:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.963961:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760778.963988:0:6505:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608320 00000100:00100000:1.0:1789760778.963995:0:6505:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608320 00000100:00100000:1.0:1789760778.964044:0:6505:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1789760778.964053:0:6505:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3637:x1876699523608320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.964103:0:6505:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3637:x1876699523608320:12345-0@lo:400:kworker.0 Request processed in 50us (1094us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.964110:0:6505:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1789760778.964143:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608832 00000100:00100000:1.0:1789760778.964147:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608832 00000100:00100000:1.0:1789760778.964171:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608576 00000100:00100000:1.0:1789760778.964174:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608576 00000100:00100000:1.0:1789760778.964188:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:1.0:1789760778.964194:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523608832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.964229:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523608832:12345-0@lo:400:kworker.0 Request processed in 34us (1141us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.964238:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:1.0:1789760778.964250:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:1.0:1789760778.964256:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523608576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.964289:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523608576:12345-0@lo:400:kworker.0 Request processed in 33us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.964294:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:2.0:1789760778.964475:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608704 00000100:00100000:2.0:1789760778.964479:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608704 00000100:00100000:2.0:1789760778.964521:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608448 00000100:00100000:2.0:1789760778.964525:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608448 00000100:00100000:2.0:1789760778.964537:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523608960 00000100:00100000:2.0:1789760778.964539:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523608960 00000100:00100000:2.0:1789760778.964560:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609472 00000100:00100000:2.0:1789760778.964564:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609472 00000100:00100000:2.0:1789760778.964576:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609088 00000100:00100000:2.0:1789760778.964579:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609088 00000100:00100000:2.0:1789760778.964601:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609600 00000100:00100000:2.0:1789760778.964605:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609600 00000100:00100000:2.0:1789760778.964621:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 965 00000100:00100000:2.0:1789760778.964637:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3637:x1876699523608704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.964675:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3637:x1876699523608704:12345-0@lo:400:kworker.0 Request processed in 48us (1445us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.964682:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 965 00000100:00100000:2.0:1789760778.964694:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 966 00000100:00100000:2.0:1789760778.964699:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3638:x1876699523608448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.964724:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3638:x1876699523608448:12345-0@lo:400:kworker.0 Request processed in 25us (1363us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.964729:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 966 00000100:00100000:2.0:1789760778.964740:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 967 00000100:00100000:2.0:1789760778.964746:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3638:x1876699523608960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.964771:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3638:x1876699523608960:12345-0@lo:400:kworker.0 Request processed in 26us (1358us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.964776:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 967 00000100:00100000:2.0:1789760778.964786:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 968 00000100:00100000:2.0:1789760778.964792:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3638:x1876699523609472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.964818:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8febb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3638:x1876699523609472:12345-0@lo:400:kworker.0 Request processed in 26us (1363us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.964824:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 968 00000100:00100000:2.0:1789760778.964834:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 969 00000100:00100000:2.0:1789760778.964840:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3638:x1876699523609088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.964866:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3638:x1876699523609088:12345-0@lo:400:kworker.0 Request processed in 27us (944us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.964872:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 969 00000100:00100000:2.0:1789760778.964883:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 970 00000100:00100000:2.0:1789760778.964888:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3638:x1876699523609600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.964914:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3638:x1876699523609600:12345-0@lo:400:kworker.0 Request processed in 26us (953us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.964920:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 970 00000100:00100000:2.0:1789760778.964945:0:8430:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609344 00000100:00100000:2.0:1789760778.964948:0:8430:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609344 00000100:00100000:2.0:1789760778.964975:0:8430:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609856 00000100:00100000:2.0:1789760778.964978:0:8430:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609856 00000100:00100000:2.0:1789760778.964990:0:8430:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609216 00000100:00100000:2.0:1789760778.964992:0:8430:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609216 00000100:00100000:2.0:1789760778.965001:0:8430:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609728 00000100:00100000:2.0:1789760778.965004:0:8430:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609728 00000100:00100000:2.0:1789760778.965015:0:8430:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:2.0:1789760778.965020:0:8430:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523609344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965053:0:8430:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523609344:12345-0@lo:400:kworker.0 Request processed in 33us (1923us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.965059:0:8430:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00100000:2.0:1789760778.965070:0:8430:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:2.0:1789760778.965076:0:8430:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523609856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965104:0:8430:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523609856:12345-0@lo:400:kworker.0 Request processed in 28us (1934us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.965115:0:8430:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:2.0:1789760778.965127:0:8430:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 311 00000100:00100000:2.0:1789760778.965134:0:8430:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523609216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965167:0:8430:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9338e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523609216:12345-0@lo:400:kworker.0 Request processed in 33us (1896us total) trans 0 rc 0/0 00000001:00080000:3.0:1789760778.965169:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986075 is committed 00000100:00100000:2.0:1789760778.965176:0:8430:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 311 00000100:00100000:2.0:1789760778.965192:0:8430:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:2.0:1789760778.965198:0:8430:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523609728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965244:0:8430:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9339180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523609728:12345-0@lo:400:kworker.0 Request processed in 46us (1936us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.965249:0:8430:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00080000:2.0:1789760778.965275:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760778.965279:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760778.965282:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760778.965284:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760778.965335:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523609984 00000100:00100000:2.0:1789760778.965339:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523609984 00000100:00100000:2.0:1789760778.965369:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1789760778.965376:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523609984:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760778.965418:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760778.965450:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523609984:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (1945us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760778.965462:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789760778.965502:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc876e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523610112:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760778.965526:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cfbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523608576:0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965545:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3638:1876699523609088:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.965546:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760778.965562:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3638:1876699523609600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965577:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cf8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3638:1876699523608448:0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965602:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3638:1876699523608960:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.965605:0:8432:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523610112 00000100:00100000:1.0:1789760778.965610:0:8432:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523610112 00000100:00100000:2.0:1789760778.965616:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3638:1876699523609472:0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965636:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523609984:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760778.965679:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cf9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3637:1876699523608704:0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965694:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523609216:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.965706:0:8432:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1789760778.965708:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523609728:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.965718:0:8432:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523610112:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760778.965722:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cf9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:afc717a1-11ba-4ef0-ab10-103aa692ca98:3637:1876699523608320:0@lo:400:kworker.0 00002000:00000024:1.0:1789760778.965732:0:8432:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760778.965737:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523608832:0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965752:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523609344:0@lo:400:kworker.0 00000100:00100000:2.0:1789760778.965765:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523609856:0@lo:400:kworker.0 00000100:00100000:1.0:1789760778.965773:0:8432:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523610112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 58us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.965781:0:8432:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:1.0:1789760778.965819:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc876e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523610112:0@lo:13:osp-pre-1-0.0 00002000:00010000:2.0:1789760778.965849:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76873728 -> 76877824 00002000:00010000:2.0:1789760778.965855:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150168 -> 150176 00010000:00010000:2.0:1789760778.965867:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.965879:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.965888:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.965896:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760778.965901:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.965910:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.965934:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.965942:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0939 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76873728->76877823] (req 76873728->76877823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.966010:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9067us (9323us total) trans 4294986075 rc 0/0 00000100:00100000:2.0:1789760778.966018:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18779 00000100:00100000:2.0:1789760778.966027:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709073311805 time 709081536873 state 0x2 ref 0 00000100:00100000:3.0:1789760778.969836:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760778.970040:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790720 00000100:00100000:0.0:1789760778.970051:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790720 00000100:00100000:0.0:1789760778.970104:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709073311805 time 709081536873 state 0x0 ref 1 00000100:00100000:0.0:1789760778.970123:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709081797289 time 709081536873 state 0x1 ref 1 00000100:00100000:0.0:1789760778.970158:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709081797289 time 709094764238 state 0x0 ref 1 00000100:00100000:0.0:1789760778.970171:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760778.970217:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76877824, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24aa0 00010000:00010000:0.0:1789760778.970237:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.970253:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760778.970258:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.970264:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.970279:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.975604:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986076 is committed 00002000:00010000:0.0:1789760778.979212:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76877824 -> 76881920 00002000:00010000:0.0:1789760778.979223:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150176 -> 150184 00010000:00010000:0.0:1789760778.979237:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.979251:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.979258:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.979265:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760778.979269:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.979277:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.979294:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.979301:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0940 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76877824->76881919] (req 76877824->76881919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.979415:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9242us (9585us total) trans 4294986076 rc 0/0 00000100:00100000:0.0:1789760778.979425:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18780 00000100:00100000:0.0:1789760778.979436:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709081797289 time 709094764238 state 0x2 ref 0 00000100:00100000:1.0:1789760778.982568:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760778.982775:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790848 00000100:00100000:3.0:1789760778.982786:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790848 00000100:00100000:3.0:1789760778.982844:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709081797289 time 709094764238 state 0x0 ref 1 00000100:00100000:3.0:1789760778.982862:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709095024654 time 709094764238 state 0x1 ref 1 00000100:00100000:3.0:1789760778.983062:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709095024654 time 709107667021 state 0x0 ref 1 00000100:00100000:3.0:1789760778.983076:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503790848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760778.983200:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76881920, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24aa8 00010000:00010000:3.0:1789760778.983211:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.983227:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760778.983231:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.983238:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.983252:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.990999:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986077 is committed 00002000:00010000:3.0:1789760778.991134:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76881920 -> 76886016 00002000:00010000:3.0:1789760778.991140:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150184 -> 150192 00010000:00010000:3.0:1789760778.991154:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991169:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991179:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991187:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760778.991192:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991202:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991224:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991232:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f0947 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76881920->76886015] (req 76881920->76886015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.992038:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503790848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8957us (9473us total) trans 4294986077 rc 0/0 00000100:00100000:3.0:1789760778.992053:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18781 00000100:00100000:3.0:1789760778.992064:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709095024654 time 709107667021 state 0x2 ref 0 00000100:00100000:1.0:1789760778.996028:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760778.997904:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503790976 00000100:00100000:2.0:1789760778.997919:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503790976 00000100:00100000:2.0:1789760778.997985:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709095024654 time 709107667021 state 0x0 ref 1 00000100:00100000:2.0:1789760778.998001:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709107927437 time 709107667021 state 0x1 ref 1 00000100:00100000:2.0:1789760778.998095:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709107927437 time 709122699839 state 0x0 ref 1 00000100:00100000:2.0:1789760778.998110:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760778.998158:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76886016, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ab0 00010000:00010000:2.0:1789760778.998166:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.998182:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760778.998188:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.998195:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.998209:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.005422:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986078 is committed 00002000:00010000:2.0:1789760779.005867:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76886016 -> 76890112 00002000:00010000:2.0:1789760779.005872:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150192 -> 150200 00010000:00010000:2.0:1789760779.005885:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.005898:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.005908:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.005915:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.005920:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.005930:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.005948:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.005961:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f094e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76886016->76890111] (req 76886016->76890111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.006079:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503790976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7968us (10057us total) trans 4294986078 rc 0/0 00000100:00100000:2.0:1789760779.006089:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18782 00000100:00100000:2.0:1789760779.006098:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709107927437 time 709122699839 state 0x2 ref 0 00000100:00100000:0.0:1789760779.008861:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.008907:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503791104 00000100:00100000:2.0:1789760779.008917:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503791104 00000100:00100000:2.0:1789760779.008971:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709107927437 time 709122699839 state 0x0 ref 1 00000100:00100000:2.0:1789760779.008985:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709122960255 time 709122699839 state 0x1 ref 1 00000100:00100000:2.0:1789760779.009016:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709122960255 time 709133621949 state 0x0 ref 1 00000100:00100000:2.0:1789760779.009028:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503791104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.009073:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76890112, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ab8 00010000:00010000:2.0:1789760779.009082:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.009098:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.009103:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.009109:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.009133:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.014902:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986079 is committed 00002000:00010000:3.0:1789760779.015091:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76890112 -> 76894208 00002000:00010000:3.0:1789760779.015097:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150200 -> 150208 00010000:00010000:3.0:1789760779.015110:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.015122:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.015137:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.015144:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.015149:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.015158:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.015175:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.015184:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0955 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76890112->76894207] (req 76890112->76894207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.015280:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503791104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6250us (6424us total) trans 4294986079 rc 0/0 00000100:00100000:3.0:1789760779.015289:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18783 00000100:00100000:3.0:1789760779.015298:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709122960255 time 709133621949 state 0x2 ref 0 00000100:00100000:1.0:1789760779.019333:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.019443:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503791232 00000100:00100000:0.0:1789760779.019454:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503791232 00000100:00100000:0.0:1789760779.019511:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709122960255 time 709133621949 state 0x0 ref 1 00000100:00100000:0.0:1789760779.019527:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709133882365 time 709133621949 state 0x1 ref 1 00000100:00100000:0.0:1789760779.019556:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709133882365 time 709144163196 state 0x0 ref 1 00000100:00100000:0.0:1789760779.019569:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.019626:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76894208, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ac0 00010000:00010000:0.0:1789760779.019636:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.019650:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.019654:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.019661:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.019674:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.025076:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986080 is committed 00002000:00010000:0.0:1789760779.025182:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76894208 -> 76898304 00002000:00010000:0.0:1789760779.025188:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150208 -> 150216 00010000:00010000:0.0:1789760779.025200:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.025213:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.025222:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.025230:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.025236:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.025245:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.025264:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.025271:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f095c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76894208->76898303] (req 76894208->76898303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.025369:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5797us (6041us total) trans 4294986080 rc 0/0 00000100:00100000:0.0:1789760779.025379:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18784 00000100:00100000:0.0:1789760779.025418:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709133882365 time 709144163196 state 0x2 ref 0 00000100:00100000:1.0:1789760779.030961:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.031061:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503791360 00000100:00100000:3.0:1789760779.031071:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503791360 00000100:00100000:3.0:1789760779.031208:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709133882365 time 709144163196 state 0x0 ref 1 00000100:00100000:3.0:1789760779.031224:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709144423612 time 709144163196 state 0x1 ref 1 00000100:00100000:3.0:1789760779.031268:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709144423612 time 709155874245 state 0x0 ref 1 00000100:00100000:3.0:1789760779.031281:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.032289:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76898304, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ac8 00010000:00010000:3.0:1789760779.032302:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.032781:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.032786:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.032793:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.032809:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.038909:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986081 is committed 00002000:00010000:3.0:1789760779.039133:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76898304 -> 76902400 00002000:00010000:3.0:1789760779.039139:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150216 -> 150224 00010000:00010000:3.0:1789760779.039152:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.039163:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.039171:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.039183:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.039187:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.039196:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.039215:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.039222:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0963 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76898304->76902399] (req 76898304->76902399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.039649:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8363us (8691us total) trans 4294986081 rc 0/0 00000100:00100000:3.0:1789760779.039661:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18785 00000100:00100000:3.0:1789760779.039672:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709144423612 time 709155874245 state 0x2 ref 0 00000100:00100000:0.0:1789760779.046296:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.046558:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503791488 00000100:00100000:1.0:1789760779.046569:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503791488 00000100:00100000:1.0:1789760779.046646:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709144423612 time 709155874245 state 0x0 ref 1 00000100:00100000:1.0:1789760779.046662:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709156134661 time 709155874245 state 0x1 ref 1 00000100:00100000:1.0:1789760779.046806:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709156134661 time 709171410247 state 0x0 ref 1 00000100:00100000:1.0:1789760779.046820:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503791488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.046873:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76902400, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ad0 00010000:00010000:1.0:1789760779.046883:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.046899:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.046904:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.046911:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.046925:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.052880:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986082 is committed 00002000:00010000:1.0:1789760779.052995:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76902400 -> 76906496 00002000:00010000:1.0:1789760779.053000:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150224 -> 150232 00010000:00010000:1.0:1789760779.053013:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.053025:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.053033:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.053040:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.053045:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.053055:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.053083:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.053089:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f096a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76902400->76906495] (req 76902400->76906495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.053166:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503791488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6345us (7857us total) trans 4294986082 rc 0/0 00000100:00100000:1.0:1789760779.053176:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18786 00000100:00100000:1.0:1789760779.053185:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709156134661 time 709171410247 state 0x2 ref 0 00000100:00100000:1.0:1789760779.055886:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.056027:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503791616 00000100:00100000:3.0:1789760779.056036:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503791616 00000100:00100000:3.0:1789760779.056085:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709156134661 time 709171410247 state 0x0 ref 1 00000100:00100000:3.0:1789760779.056099:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709171670663 time 709171410247 state 0x1 ref 1 00000100:00100000:3.0:1789760779.056155:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709171670663 time 709180761426 state 0x0 ref 1 00000100:00100000:3.0:1789760779.056169:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503791616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.056215:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76906496, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ad8 00010000:00010000:3.0:1789760779.056225:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056245:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.056250:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.056256:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056270:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.061769:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986083 is committed 00002000:00010000:2.0:1789760779.061884:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76906496 -> 76910592 00002000:00010000:2.0:1789760779.061890:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150232 -> 150240 00010000:00010000:2.0:1789760779.061902:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.061915:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.061924:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.061931:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.061935:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.061944:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.061962:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.061970:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f0971 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76906496->76910591] (req 76906496->76910591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.062102:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503791616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5930us (6219us total) trans 4294986083 rc 0/0 00000100:00100000:2.0:1789760779.062112:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18787 00000100:00100000:2.0:1789760779.062122:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709171670663 time 709180761426 state 0x2 ref 0 00000100:00100000:2.0:1789760779.067096:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.067167:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503791744 00000100:00100000:0.0:1789760779.067177:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503791744 00000100:00100000:0.0:1789760779.067231:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709171670663 time 709180761426 state 0x0 ref 1 00000100:00100000:0.0:1789760779.067245:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709181021842 time 709180761426 state 0x1 ref 1 00000100:00100000:0.0:1789760779.067279:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709181021842 time 709191885153 state 0x0 ref 1 00000100:00100000:0.0:1789760779.067296:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.067347:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76910592, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ae0 00010000:00010000:0.0:1789760779.067356:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.067370:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.067375:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.067381:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.067434:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.075302:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986084 is committed 00002000:00010000:0.0:1789760779.075478:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76910592 -> 76914688 00002000:00010000:0.0:1789760779.075484:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150240 -> 150248 00010000:00010000:0.0:1789760779.075496:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.075508:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.075517:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.075523:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.075528:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.075537:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.075557:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.075565:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0978 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76910592->76914687] (req 76910592->76914687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.076089:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8790us (8996us total) trans 4294986084 rc 0/0 00000100:00100000:0.0:1789760779.076099:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18788 00000100:00100000:0.0:1789760779.076110:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709181021842 time 709191885153 state 0x2 ref 0 00000100:00100000:0.0:1789760779.079952:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.080048:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503791872 00000100:00100000:1.0:1789760779.080059:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503791872 00000100:00100000:1.0:1789760779.080113:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709181021842 time 709191885153 state 0x0 ref 1 00000100:00100000:1.0:1789760779.080131:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709192145569 time 709191885153 state 0x1 ref 1 00000100:00100000:1.0:1789760779.080165:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709192145569 time 709204771010 state 0x0 ref 1 00000100:00100000:1.0:1789760779.080176:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.080219:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76914688, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ae8 00010000:00010000:1.0:1789760779.080228:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.080244:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.080249:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.080255:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.080269:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.085434:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986085 is committed 00002000:00010000:1.0:1789760779.085803:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76914688 -> 76918784 00002000:00010000:1.0:1789760779.085808:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150248 -> 150256 00010000:00010000:1.0:1789760779.085822:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.085834:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.085842:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.085850:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.085855:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.085864:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.085884:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.085892:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f097f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76914688->76918783] (req 76914688->76918783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.085989:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503791872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5809us (6042us total) trans 4294986085 rc 0/0 00000100:00100000:1.0:1789760779.085999:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18789 00000100:00100000:1.0:1789760779.086009:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709192145569 time 709204771010 state 0x2 ref 0 00000100:00100000:1.0:1789760779.092529:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.092623:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792000 00000100:00100000:2.0:1789760779.092632:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792000 00000100:00100000:2.0:1789760779.092681:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709192145569 time 709204771010 state 0x0 ref 1 00000100:00100000:2.0:1789760779.092696:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709205031426 time 709204771010 state 0x1 ref 1 00000100:00100000:2.0:1789760779.092729:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709205031426 time 709217335282 state 0x0 ref 1 00000100:00100000:2.0:1789760779.092741:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503792000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.092797:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76918784, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24af0 00010000:00010000:2.0:1789760779.092807:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.092822:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.092826:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.092833:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.092846:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.098428:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986086 is committed 00002000:00010000:2.0:1789760779.098696:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76918784 -> 76922880 00002000:00010000:2.0:1789760779.098701:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150256 -> 150264 00010000:00010000:2.0:1789760779.098713:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098727:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098736:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098743:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.098747:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098757:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098774:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098782:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0986 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76918784->76922879] (req 76918784->76922879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.098988:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503792000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6245us (6465us total) trans 4294986086 rc 0/0 00000100:00100000:2.0:1789760779.099002:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18790 00000100:00100000:2.0:1789760779.099012:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709205031426 time 709217335282 state 0x2 ref 0 00000100:00100000:2.0:1789760779.111957:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.112851:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792128 00000100:00100000:1.0:1789760779.112862:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792128 00000100:00100000:1.0:1789760779.112998:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709205031426 time 709217335282 state 0x0 ref 1 00000100:00100000:1.0:1789760779.113013:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709217595698 time 709217335282 state 0x1 ref 1 00000100:00100000:1.0:1789760779.113159:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709217595698 time 709237763607 state 0x0 ref 1 00000100:00100000:1.0:1789760779.113173:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503792128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.113221:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76922880, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24af8 00010000:00010000:1.0:1789760779.113230:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.113244:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.113249:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.113254:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.113267:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.121052:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986087 is committed 00002000:00010000:1.0:1789760779.122792:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76922880 -> 76926976 00002000:00010000:1.0:1789760779.122800:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150264 -> 150272 00010000:00010000:1.0:1789760779.122814:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.122827:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.122837:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.122844:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.122850:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.122859:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.122878:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.122886:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f098d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76922880->76926975] (req 76922880->76926975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.123116:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503792128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9940us (11163us total) trans 4294986087 rc 0/0 00000100:00100000:1.0:1789760779.123126:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18791 00000100:00100000:1.0:1789760779.123146:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709217595698 time 709237763607 state 0x2 ref 0 00000100:00100000:0.0:1789760779.127760:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.127901:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792256 00000100:00100000:1.0:1789760779.127911:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792256 00000100:00100000:1.0:1789760779.127962:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709217595698 time 709237763607 state 0x0 ref 1 00000100:00100000:1.0:1789760779.127977:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709238024023 time 709237763607 state 0x1 ref 1 00000100:00100000:1.0:1789760779.128012:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709238024023 time 709252617831 state 0x0 ref 1 00000100:00100000:1.0:1789760779.128025:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503792256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.128070:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76926976, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b00 00010000:00010000:1.0:1789760779.128080:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.128097:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.128102:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.128109:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.128123:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.135198:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986088 is committed 00002000:00010000:1.0:1789760779.135325:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76926976 -> 76931072 00002000:00010000:1.0:1789760779.135331:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150272 -> 150280 00010000:00010000:1.0:1789760779.135344:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.135357:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.135365:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.135373:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.135378:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.135386:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.135424:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.135431:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0994 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76926976->76931071] (req 76926976->76931071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.135525:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503792256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7498us (7770us total) trans 4294986088 rc 0/0 00000100:00100000:1.0:1789760779.135536:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18792 00000100:00100000:1.0:1789760779.135546:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709238024023 time 709252617831 state 0x2 ref 0 00000100:00100000:2.0:1789760779.139806:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.139853:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792384 00000100:00100000:3.0:1789760779.139864:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792384 00000100:00100000:3.0:1789760779.139925:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709238024023 time 709252617831 state 0x0 ref 1 00000100:00100000:3.0:1789760779.140211:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709252878247 time 709252617831 state 0x1 ref 1 00000100:00100000:3.0:1789760779.140243:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709252878247 time 709264849979 state 0x0 ref 1 00000100:00100000:3.0:1789760779.140264:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503792384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.140311:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76931072, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b08 00010000:00010000:3.0:1789760779.140320:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.140335:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.140339:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.140345:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.140357:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.148575:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986089 is committed 00002000:00010000:2.0:1789760779.149466:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76931072 -> 76935168 00002000:00010000:2.0:1789760779.149474:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150280 -> 150288 00010000:00010000:2.0:1789760779.149488:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.149502:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.149516:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.149526:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.149532:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.149542:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.149563:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.149576:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f099b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76931072->76935167] (req 76931072->76935167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.149694:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503792384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9436us (9893us total) trans 4294986089 rc 0/0 00000100:00100000:2.0:1789760779.149705:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18793 00000100:00100000:2.0:1789760779.149714:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709252878247 time 709264849979 state 0x2 ref 0 00000100:00100000:2.0:1789760779.153828:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.153904:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792512 00000100:00100000:3.0:1789760779.153914:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792512 00000100:00100000:3.0:1789760779.153969:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709252878247 time 709264849979 state 0x0 ref 1 00000100:00100000:3.0:1789760779.153984:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709265110395 time 709264849979 state 0x1 ref 1 00000100:00100000:3.0:1789760779.154019:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709265110395 time 709278624970 state 0x0 ref 1 00000100:00100000:3.0:1789760779.154032:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503792512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.154079:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76935168, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b10 00010000:00010000:3.0:1789760779.154089:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.154106:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.154110:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.154117:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.154132:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.162787:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986090 is committed 00002000:00010000:3.0:1789760779.162994:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76935168 -> 76939264 00002000:00010000:3.0:1789760779.163000:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150288 -> 150296 00010000:00010000:3.0:1789760779.163012:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.163024:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.163033:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.163040:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.163045:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.163054:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.163073:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.163080:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76935168->76939263] (req 76935168->76939263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.163573:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503792512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9537us (9748us total) trans 4294986090 rc 0/0 00000100:00100000:3.0:1789760779.163584:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18794 00000100:00100000:3.0:1789760779.163603:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709265110395 time 709278624970 state 0x2 ref 0 00000100:00100000:0.0:1789760779.168730:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.168836:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792640 00000100:00100000:1.0:1789760779.168846:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792640 00000100:00100000:1.0:1789760779.169131:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709265110395 time 709278624970 state 0x0 ref 1 00000100:00100000:1.0:1789760779.169147:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709278885386 time 709278624970 state 0x1 ref 1 00000100:00100000:1.0:1789760779.169306:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709278885386 time 709293911003 state 0x0 ref 1 00000100:00100000:1.0:1789760779.169560:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503792640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.169625:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76939264, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b18 00010000:00010000:1.0:1789760779.169634:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.169650:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.169654:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.169660:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.169674:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.178360:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986091 is committed 00002000:00010000:1.0:1789760779.179497:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76939264 -> 76943360 00002000:00010000:1.0:1789760779.179506:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150296 -> 150304 00010000:00010000:1.0:1789760779.179522:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.179536:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.179546:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.179555:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.179560:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.179571:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.179603:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.179618:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76939264->76943359] (req 76939264->76943359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.179718:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503792640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10157us (10990us total) trans 4294986091 rc 0/0 00000100:00100000:1.0:1789760779.179729:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18795 00000100:00100000:1.0:1789760779.179739:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709278885386 time 709293911003 state 0x2 ref 0 00000100:00100000:0.0:1789760779.185798:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.186040:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792768 00000100:00100000:1.0:1789760779.186051:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792768 00000100:00100000:1.0:1789760779.186103:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709278885386 time 709293911003 state 0x0 ref 1 00000100:00100000:1.0:1789760779.186119:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709294171419 time 709293911003 state 0x1 ref 1 00000100:00100000:1.0:1789760779.186155:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709294171419 time 709310761378 state 0x0 ref 1 00000100:00100000:1.0:1789760779.186169:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503792768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.186213:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76943360, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b20 00010000:00010000:1.0:1789760779.186222:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.186238:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.186242:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.186248:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.186268:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.195844:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986092 is committed 00002000:00010000:1.0:1789760779.196279:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76943360 -> 76947456 00002000:00010000:1.0:1789760779.196285:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150304 -> 150312 00010000:00010000:1.0:1789760779.196299:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.196311:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.196319:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.196326:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.196331:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.196341:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.196362:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.196369:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76943360->76947455] (req 76943360->76947455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.196511:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503792768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10338us (10717us total) trans 4294986092 rc 0/0 00000100:00100000:1.0:1789760779.196522:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18796 00000100:00100000:1.0:1789760779.196533:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709294171419 time 709310761378 state 0x2 ref 0 00000100:00100000:2.0:1789760779.199732:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.199816:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503792896 00000100:00100000:1.0:1789760779.199826:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503792896 00000100:00100000:1.0:1789760779.199882:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709294171419 time 709310761378 state 0x0 ref 1 00000100:00100000:1.0:1789760779.199897:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709311021794 time 709310761378 state 0x1 ref 1 00000100:00100000:1.0:1789760779.199928:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709311021794 time 709324533871 state 0x0 ref 1 00000100:00100000:1.0:1789760779.199941:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503792896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.199989:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76947456, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b28 00010000:00010000:1.0:1789760779.199998:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200014:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.200019:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.200025:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200038:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.207484:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986093 is committed 00002000:00010000:1.0:1789760779.207618:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76947456 -> 76951552 00002000:00010000:1.0:1789760779.207624:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150312 -> 150320 00010000:00010000:1.0:1789760779.207636:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.207648:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.207656:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.207663:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.207667:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.207676:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.207694:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.207701:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f09b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76947456->76951551] (req 76947456->76951551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.208205:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503792896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8262us (8479us total) trans 4294986093 rc 0/0 00000100:00100000:1.0:1789760779.208215:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18797 00000100:00100000:1.0:1789760779.208226:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709311021794 time 709324533871 state 0x2 ref 0 00000100:00100000:0.0:1789760779.215124:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.215544:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793024 00000100:00100000:3.0:1789760779.215554:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793024 00000100:00100000:3.0:1789760779.215623:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709311021794 time 709324533871 state 0x0 ref 1 00000100:00100000:3.0:1789760779.215649:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709324794287 time 709324533871 state 0x1 ref 1 00000100:00100000:3.0:1789760779.215679:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709324794287 time 709340285935 state 0x0 ref 1 00000100:00100000:3.0:1789760779.215690:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.215738:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76951552, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b30 00010000:00010000:3.0:1789760779.215748:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.215772:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.215776:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.215782:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.215802:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.223677:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986094 is committed 00002000:00010000:2.0:1789760779.230202:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76951552 -> 76955648 00002000:00010000:2.0:1789760779.230210:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150320 -> 150328 00010000:00010000:2.0:1789760779.230222:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.230233:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.230242:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.230249:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.230254:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.230269:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.230288:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.230296:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f09be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76951552->76955647] (req 76951552->76955647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.230727:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15034us (15609us total) trans 4294986094 rc 0/0 00000100:00100000:2.0:1789760779.230739:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18798 00000100:00100000:2.0:1789760779.230748:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709324794287 time 709340285935 state 0x2 ref 0 00000100:00100000:0.0:1789760779.234782:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.235140:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793152 00000100:00100000:2.0:1789760779.235150:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793152 00000100:00100000:2.0:1789760779.235204:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709324794287 time 709340285935 state 0x0 ref 1 00000100:00100000:2.0:1789760779.235219:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709340546351 time 709340285935 state 0x1 ref 1 00000100:00100000:2.0:1789760779.235298:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709340546351 time 709359903550 state 0x0 ref 1 00000100:00100000:2.0:1789760779.235322:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.235367:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76955648, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b38 00010000:00010000:2.0:1789760779.235376:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.235430:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.235435:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.235441:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.235455:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.243529:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986095 is committed 00002000:00010000:2.0:1789760779.243736:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76955648 -> 76959744 00002000:00010000:2.0:1789760779.243742:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150328 -> 150336 00010000:00010000:2.0:1789760779.243755:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.243767:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.243776:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.243784:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.243789:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.243797:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.243820:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.243826:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76955648->76959743] (req 76955648->76959743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.243992:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8679us (9215us total) trans 4294986095 rc 0/0 00000100:00100000:2.0:1789760779.244001:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18799 00000100:00100000:2.0:1789760779.244011:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709340546351 time 709359903550 state 0x2 ref 0 00000100:00100000:1.0:1789760779.246867:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.246973:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793280 00000100:00100000:0.0:1789760779.246983:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793280 00000100:00100000:0.0:1789760779.247032:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709340546351 time 709359903550 state 0x0 ref 1 00000100:00100000:0.0:1789760779.247047:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709360163966 time 709359903550 state 0x1 ref 1 00000100:00100000:0.0:1789760779.247080:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709360163966 time 709371686395 state 0x0 ref 1 00000100:00100000:0.0:1789760779.247091:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.247138:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76959744, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b40 00010000:00010000:0.0:1789760779.247146:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.247161:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.247166:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.247173:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.247186:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.260829:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986096 is committed 00002000:00010000:1.0:1789760779.261342:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76959744 -> 76963840 00002000:00010000:1.0:1789760779.261348:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150336 -> 150344 00010000:00010000:1.0:1789760779.261362:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.261374:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.261383:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.261432:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.261437:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.261447:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.261467:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.261475:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f09cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76959744->76963839] (req 76959744->76963839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.261571:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14475us (14708us total) trans 4294986096 rc 0/0 00000100:00100000:1.0:1789760779.261582:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18800 00000100:00100000:1.0:1789760779.261608:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709360163966 time 709371686395 state 0x2 ref 0 00000100:00100000:0.0:1789760779.266650:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.266722:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793408 00000100:00100000:1.0:1789760779.266733:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793408 00000100:00100000:1.0:1789760779.266788:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709360163966 time 709371686395 state 0x0 ref 1 00000100:00100000:1.0:1789760779.266804:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709371946811 time 709371686395 state 0x1 ref 1 00000100:00100000:1.0:1789760779.266836:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709371946811 time 709391441683 state 0x0 ref 1 00000100:00100000:1.0:1789760779.266850:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.266898:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76963840, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b48 00010000:00010000:1.0:1789760779.266907:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266922:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.266927:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.266933:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.266946:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.273886:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986097 is committed 00002000:00010000:2.0:1789760779.274012:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76963840 -> 76967936 00002000:00010000:2.0:1789760779.274018:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150344 -> 150352 00010000:00010000:2.0:1789760779.274032:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.274044:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.274053:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.274061:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.274065:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.274080:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.274099:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.274107:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f09d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76963840->76967935] (req 76963840->76967935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.274197:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7345us (7720us total) trans 4294986097 rc 0/0 00000100:00100000:2.0:1789760779.274207:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18801 00000100:00100000:2.0:1789760779.274216:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709371946811 time 709391441683 state 0x2 ref 0 00000100:00100000:0.0:1789760779.277557:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.277921:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793536 00000100:00100000:2.0:1789760779.277931:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793536 00000100:00100000:2.0:1789760779.277991:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709371946811 time 709391441683 state 0x0 ref 1 00000100:00100000:2.0:1789760779.278007:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709391702099 time 709391441683 state 0x1 ref 1 00000100:00100000:2.0:1789760779.278040:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709391702099 time 709402646571 state 0x0 ref 1 00000100:00100000:2.0:1789760779.278053:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.278102:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76967936, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b50 00010000:00010000:2.0:1789760779.278111:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.278128:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.278132:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.278139:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.278152:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.289703:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986098 is committed 00002000:00010000:2.0:1789760779.289844:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76967936 -> 76972032 00002000:00010000:2.0:1789760779.289849:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150352 -> 150360 00010000:00010000:2.0:1789760779.289861:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.289873:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.289882:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.289890:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.289894:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.289902:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.289927:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.289935:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76967936->76972031] (req 76967936->76972031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.290033:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11977us (12481us total) trans 4294986098 rc 0/0 00000100:00100000:2.0:1789760779.290042:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18802 00000100:00100000:2.0:1789760779.290052:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709391702099 time 709402646571 state 0x2 ref 0 00000100:00100000:1.0:1789760779.294791:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.294893:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793664 00000100:00100000:2.0:1789760779.294903:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793664 00000100:00100000:2.0:1789760779.294960:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709391702099 time 709402646571 state 0x0 ref 1 00000100:00100000:2.0:1789760779.294976:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709402906987 time 709402646571 state 0x1 ref 1 00000100:00100000:2.0:1789760779.295011:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709402906987 time 709419617296 state 0x0 ref 1 00000100:00100000:2.0:1789760779.295024:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.295072:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76972032, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b58 00010000:00010000:2.0:1789760779.295081:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.295096:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.295101:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.295108:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.295122:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.306435:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986099 is committed 00002000:00010000:1.0:1789760779.306560:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76972032 -> 76976128 00002000:00010000:1.0:1789760779.306565:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150360 -> 150368 00010000:00010000:1.0:1789760779.306578:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.306603:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.306612:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.306618:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.306623:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.306638:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.306657:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.306663:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f09e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76972032->76976127] (req 76972032->76976127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.306765:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11739us (11979us total) trans 4294986099 rc 0/0 00000100:00100000:1.0:1789760779.306776:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18803 00000100:00100000:1.0:1789760779.306786:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709402906987 time 709419617296 state 0x2 ref 0 00000100:00100000:0.0:1789760779.311747:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.311910:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793792 00000100:00100000:1.0:1789760779.311920:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793792 00000100:00100000:1.0:1789760779.311976:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709402906987 time 709419617296 state 0x0 ref 1 00000100:00100000:1.0:1789760779.311991:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709419877712 time 709419617296 state 0x1 ref 1 00000100:00100000:1.0:1789760779.312024:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709419877712 time 709436629760 state 0x0 ref 1 00000100:00100000:1.0:1789760779.312036:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.312085:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76976128, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b60 00010000:00010000:1.0:1789760779.312094:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.312110:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.312114:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.312120:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.312134:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.318806:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986100 is committed 00002000:00010000:0.0:1789760779.319341:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76976128 -> 76980224 00002000:00010000:0.0:1789760779.319347:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150368 -> 150376 00010000:00010000:0.0:1789760779.319361:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.319374:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.319383:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322426:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.322434:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322450:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322472:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322480:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f09e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76976128->76980223] (req 76976128->76980223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.322900:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503793792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10862us (11159us total) trans 4294986100 rc 0/0 00000100:00100000:0.0:1789760779.322912:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18804 00000100:00100000:0.0:1789760779.322922:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709419877712 time 709436629760 state 0x2 ref 0 00000100:00100000:2.0:1789760779.328173:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.328279:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503793920 00000100:00100000:1.0:1789760779.328290:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503793920 00000100:00100000:1.0:1789760779.328353:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709419877712 time 709436629760 state 0x0 ref 1 00000100:00100000:1.0:1789760779.328367:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709436890176 time 709436629760 state 0x1 ref 1 00000100:00100000:1.0:1789760779.328473:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709436890176 time 709453078875 state 0x0 ref 1 00000100:00100000:1.0:1789760779.328486:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.328529:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76980224, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b68 00010000:00010000:1.0:1789760779.328539:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.328554:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.328559:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.328565:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.328578:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.337207:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986101 is committed 00002000:00010000:2.0:1789760779.337321:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76980224 -> 76984320 00002000:00010000:2.0:1789760779.337326:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150376 -> 150384 00010000:00010000:2.0:1789760779.337339:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.337352:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.337359:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.337367:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.337371:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.337379:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.337437:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.337445:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f09ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76980224->76984319] (req 76980224->76984319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.337581:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503793920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9092us (9413us total) trans 4294986101 rc 0/0 00000100:00100000:2.0:1789760779.337601:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18805 00000100:00100000:2.0:1789760779.337611:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709436890176 time 709453078875 state 0x2 ref 0 00000100:00100000:1.0:1789760779.340535:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.340794:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794048 00000100:00100000:2.0:1789760779.340805:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794048 00000100:00100000:2.0:1789760779.340859:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709436890176 time 709453078875 state 0x0 ref 1 00000100:00100000:2.0:1789760779.340874:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709453339291 time 709453078875 state 0x1 ref 1 00000100:00100000:2.0:1789760779.340903:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709453339291 time 709465509439 state 0x0 ref 1 00000100:00100000:2.0:1789760779.340915:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503794048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.340962:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76984320, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b70 00010000:00010000:2.0:1789760779.340971:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.340986:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.340991:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.340997:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.341144:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.347143:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986102 is committed 00002000:00010000:2.0:1789760779.347268:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76984320 -> 76988416 00002000:00010000:2.0:1789760779.347273:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150384 -> 150392 00010000:00010000:2.0:1789760779.347285:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.347299:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.347519:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.347529:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.347534:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.347544:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.347564:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.347573:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76984320->76988415] (req 76984320->76988415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.347719:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503794048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6801us (7188us total) trans 4294986102 rc 0/0 00000100:00100000:2.0:1789760779.347730:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18806 00000100:00100000:2.0:1789760779.347740:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709453339291 time 709465509439 state 0x2 ref 0 00000100:00100000:0.0:1789760779.354836:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.355096:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794176 00000100:00100000:2.0:1789760779.355107:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794176 00000100:00100000:2.0:1789760779.355599:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709453339291 time 709465509439 state 0x0 ref 1 00000100:00100000:2.0:1789760779.355616:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709465769855 time 709465509439 state 0x1 ref 1 00000100:00100000:2.0:1789760779.355745:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709465769855 time 709480349061 state 0x0 ref 1 00000100:00100000:2.0:1789760779.355760:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503794176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.355807:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76988416, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b78 00010000:00010000:2.0:1789760779.355817:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.355833:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.355837:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.355844:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.355868:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.361243:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986103 is committed 00002000:00010000:2.0:1789760779.361762:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76988416 -> 76992512 00002000:00010000:2.0:1789760779.361767:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150392 -> 150400 00010000:00010000:2.0:1789760779.361781:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.361795:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.361804:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.361812:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.361817:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.361826:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.361848:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.361855:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f09fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76988416->76992511] (req 76988416->76992511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.362109:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503794176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6348us (7277us total) trans 4294986103 rc 0/0 00000100:00100000:2.0:1789760779.362121:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18807 00000100:00100000:2.0:1789760779.362131:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709465769855 time 709480349061 state 0x2 ref 0 00000100:00100000:0.0:1789760779.365691:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.365776:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794304 00000100:00100000:1.0:1789760779.365786:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794304 00000100:00100000:1.0:1789760779.365843:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709465769855 time 709480349061 state 0x0 ref 1 00000100:00100000:1.0:1789760779.365858:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709480609477 time 709480349061 state 0x1 ref 1 00000100:00100000:1.0:1789760779.365889:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709480609477 time 709490494944 state 0x0 ref 1 00000100:00100000:1.0:1789760779.365903:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503794304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.365950:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76992512, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b80 00010000:00010000:1.0:1789760779.365959:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.365975:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.365980:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.365986:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.365998:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.373784:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986104 is committed 00002000:00010000:1.0:1789760779.373985:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76992512 -> 76996608 00002000:00010000:1.0:1789760779.373991:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150400 -> 150408 00010000:00010000:1.0:1789760779.374004:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.374023:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.374032:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.374040:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.374045:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.374054:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.374074:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.374082:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0a04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76992512->76996607] (req 76992512->76996607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.374228:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503794304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8323us (8542us total) trans 4294986104 rc 0/0 00000100:00100000:1.0:1789760779.374239:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18808 00000100:00100000:1.0:1789760779.374249:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709480609477 time 709490494944 state 0x2 ref 0 00000100:00100000:1.0:1789760779.381608:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.381854:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794432 00000100:00100000:2.0:1789760779.381865:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794432 00000100:00100000:2.0:1789760779.381918:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709480609477 time 709490494944 state 0x0 ref 1 00000100:00100000:2.0:1789760779.381935:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709490755360 time 709490494944 state 0x1 ref 1 00000100:00100000:2.0:1789760779.381966:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709490755360 time 709506572517 state 0x0 ref 1 00000100:00100000:2.0:1789760779.381978:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503794432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.382027:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 76996608, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b88 00010000:00010000:2.0:1789760779.382035:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.382051:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.382056:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.382062:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.382076:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.388940:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986105 is committed 00002000:00010000:2.0:1789760779.389047:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 76996608 -> 77000704 00002000:00010000:2.0:1789760779.389052:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150408 -> 150416 00010000:00010000:2.0:1789760779.389066:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.389079:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.389089:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.389096:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.389101:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.389111:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.389131:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.389139:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [76996608->77000703] (req 76996608->77000703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.389245:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503794432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7263us (7656us total) trans 4294986105 rc 0/0 00000100:00100000:2.0:1789760779.389256:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18809 00000100:00100000:2.0:1789760779.389266:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709490755360 time 709506572517 state 0x2 ref 0 00000100:00100000:0.0:1789760779.392811:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.392914:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794560 00000100:00100000:2.0:1789760779.392923:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794560 00000100:00100000:2.0:1789760779.392980:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709490755360 time 709506572517 state 0x0 ref 1 00000100:00100000:2.0:1789760779.392996:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709506832933 time 709506572517 state 0x1 ref 1 00000100:00100000:2.0:1789760779.393029:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709506832933 time 709517635592 state 0x0 ref 1 00000100:00100000:2.0:1789760779.393042:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503794560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.393091:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77000704, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b90 00010000:00010000:2.0:1789760779.393101:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.393117:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.393121:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.393127:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.393141:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.402227:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986106 is committed 00002000:00010000:3.0:1789760779.402578:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77000704 -> 77004800 00002000:00010000:3.0:1789760779.402593:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150416 -> 150424 00010000:00010000:3.0:1789760779.402607:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.402620:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.402629:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.402637:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.402642:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.402650:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.402669:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.402677:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0a12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77000704->77004799] (req 77000704->77004799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.402801:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503794560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9757us (9995us total) trans 4294986106 rc 0/0 00000100:00100000:3.0:1789760779.402812:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18810 00000100:00100000:3.0:1789760779.402831:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709506832933 time 709517635592 state 0x2 ref 0 00000100:00100000:1.0:1789760779.408096:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.408770:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794688 00000100:00100000:2.0:1789760779.408781:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794688 00000100:00100000:2.0:1789760779.408836:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709506832933 time 709517635592 state 0x0 ref 1 00000100:00100000:2.0:1789760779.408852:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709517896008 time 709517635592 state 0x1 ref 1 00000100:00100000:2.0:1789760779.408884:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709517896008 time 709533491393 state 0x0 ref 1 00000100:00100000:2.0:1789760779.408897:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503794688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.408944:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77004800, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24b98 00010000:00010000:2.0:1789760779.408954:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.408969:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.408973:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.408980:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.408992:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.414816:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986107 is committed 00002000:00010000:2.0:1789760779.415029:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77004800 -> 77008896 00002000:00010000:2.0:1789760779.415034:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150424 -> 150432 00010000:00010000:2.0:1789760779.415048:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.415061:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.415070:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.415084:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.415089:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.415098:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.415117:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.415125:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77004800->77008895] (req 77004800->77008895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.417683:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503794688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8485us (9295us total) trans 4294986107 rc 0/0 00000100:00100000:2.0:1789760779.417699:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18811 00000100:00100000:2.0:1789760779.417710:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709517896008 time 709533491393 state 0x2 ref 0 00000100:00100000:1.0:1789760779.421811:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.421904:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794816 00000100:00100000:0.0:1789760779.421915:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794816 00000100:00100000:0.0:1789760779.421971:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709517896008 time 709533491393 state 0x0 ref 1 00000100:00100000:0.0:1789760779.421986:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709533751809 time 709533491393 state 0x1 ref 1 00000100:00100000:0.0:1789760779.422018:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709533751809 time 709546624671 state 0x0 ref 1 00000100:00100000:0.0:1789760779.422039:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503794816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.422085:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77008896, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ba0 00010000:00010000:0.0:1789760779.422094:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.422109:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.422113:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.422120:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.422133:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.429377:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986108 is committed 00002000:00010000:0.0:1789760779.429622:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77008896 -> 77012992 00002000:00010000:0.0:1789760779.429628:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150432 -> 150440 00010000:00010000:0.0:1789760779.429641:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.429655:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.429664:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.429671:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.429676:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.429684:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.429704:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.429712:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0a20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77008896->77012991] (req 77008896->77012991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.429854:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503794816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7812us (8049us total) trans 4294986108 rc 0/0 00000100:00100000:0.0:1789760779.429865:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18812 00000100:00100000:0.0:1789760779.429876:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709533751809 time 709546624671 state 0x2 ref 0 00000100:00100000:0.0:1789760779.436981:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.437089:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503794944 00000100:00100000:1.0:1789760779.437099:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503794944 00000100:00100000:1.0:1789760779.437155:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709533751809 time 709546624671 state 0x0 ref 1 00000100:00100000:1.0:1789760779.437170:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709546885087 time 709546624671 state 0x1 ref 1 00000100:00100000:1.0:1789760779.437201:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709546885087 time 709561808139 state 0x0 ref 1 00000100:00100000:1.0:1789760779.437214:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503794944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.437262:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77012992, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ba8 00010000:00010000:1.0:1789760779.437271:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.437287:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.437291:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.437299:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.437322:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.442137:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986109 is committed 00002000:00010000:2.0:1789760779.442296:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77012992 -> 77017088 00002000:00010000:2.0:1789760779.442301:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150440 -> 150448 00010000:00010000:2.0:1789760779.442376:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.442428:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.442444:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.442451:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.442456:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.442464:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.442483:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.442491:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0a27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77012992->77017087] (req 77012992->77017087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.442598:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503794944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5366us (5605us total) trans 4294986109 rc 0/0 00000100:00100000:2.0:1789760779.442609:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18813 00000100:00100000:2.0:1789760779.442620:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709546885087 time 709561808139 state 0x2 ref 0 00000100:00100000:0.0:1789760779.446832:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.447221:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795072 00000100:00100000:3.0:1789760779.447230:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795072 00000100:00100000:3.0:1789760779.447280:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709546885087 time 709561808139 state 0x0 ref 1 00000100:00100000:3.0:1789760779.447295:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709562068555 time 709561808139 state 0x1 ref 1 00000100:00100000:3.0:1789760779.447332:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709562068555 time 709571938264 state 0x0 ref 1 00000100:00100000:3.0:1789760779.447346:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.447428:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77017088, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bb0 00010000:00010000:3.0:1789760779.447438:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.447453:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.447457:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.447463:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.447481:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.456364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986110 is committed 00002000:00010000:3.0:1789760779.456499:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77017088 -> 77021184 00002000:00010000:3.0:1789760779.456505:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150448 -> 150456 00010000:00010000:3.0:1789760779.456518:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456532:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456541:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456548:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.456553:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456563:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456582:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456603:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77017088->77021183] (req 77017088->77021183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.456870:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9521us (10039us total) trans 4294986110 rc 0/0 00000100:00100000:3.0:1789760779.456881:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18814 00000100:00100000:3.0:1789760779.456901:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709562068555 time 709571938264 state 0x2 ref 0 00000100:00100000:1.0:1789760779.460562:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.460630:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795200 00000100:00100000:2.0:1789760779.460640:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795200 00000100:00100000:2.0:1789760779.460694:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709562068555 time 709571938264 state 0x0 ref 1 00000100:00100000:2.0:1789760779.460709:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709572198680 time 709571938264 state 0x1 ref 1 00000100:00100000:2.0:1789760779.460745:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709572198680 time 709585351745 state 0x0 ref 1 00000100:00100000:2.0:1789760779.460757:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.460803:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77021184, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bb8 00010000:00010000:2.0:1789760779.460812:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.460833:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.460838:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.460845:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.460860:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.467243:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986111 is committed 00002000:00010000:1.0:1789760779.467378:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77021184 -> 77025280 00002000:00010000:1.0:1789760779.467383:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150456 -> 150464 00010000:00010000:1.0:1789760779.467435:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467448:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467456:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467463:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.467467:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467475:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467494:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467500:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0a35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77021184->77025279] (req 77021184->77025279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.467578:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6818us (7022us total) trans 4294986111 rc 0/0 00000100:00100000:1.0:1789760779.467600:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18815 00000100:00100000:1.0:1789760779.467610:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709572198680 time 709585351745 state 0x2 ref 0 00000100:00100000:0.0:1789760779.472638:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.472808:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795328 00000100:00100000:2.0:1789760779.472817:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795328 00000100:00100000:2.0:1789760779.472874:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709572198680 time 709585351745 state 0x0 ref 1 00000100:00100000:2.0:1789760779.472899:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709585612161 time 709585351745 state 0x1 ref 1 00000100:00100000:2.0:1789760779.472977:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709585612161 time 709597583117 state 0x0 ref 1 00000100:00100000:2.0:1789760779.472990:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.473039:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77025280, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bc0 00010000:00010000:2.0:1789760779.473048:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.473065:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.473069:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.473076:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.473090:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.483200:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986112 is committed 00002000:00010000:2.0:1789760779.483347:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77025280 -> 77029376 00002000:00010000:2.0:1789760779.483353:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150464 -> 150472 00010000:00010000:2.0:1789760779.483487:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.483503:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.483511:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.483518:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.483522:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.483537:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.483561:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.483569:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77025280->77029375] (req 77025280->77029375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.483813:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10817us (11186us total) trans 4294986112 rc 0/0 00000100:00100000:2.0:1789760779.483826:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18816 00000100:00100000:2.0:1789760779.483836:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709585612161 time 709597583117 state 0x2 ref 0 00000100:00100000:0.0:1789760779.490467:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.490605:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795456 00000100:00100000:1.0:1789760779.490616:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795456 00000100:00100000:1.0:1789760779.490670:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709585612161 time 709597583117 state 0x0 ref 1 00000100:00100000:1.0:1789760779.490697:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709597843533 time 709597583117 state 0x1 ref 1 00000100:00100000:1.0:1789760779.490732:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709597843533 time 709615338102 state 0x0 ref 1 00000100:00100000:1.0:1789760779.490744:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.490793:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77029376, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bc8 00010000:00010000:1.0:1789760779.490803:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.490819:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.490823:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.490834:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.490850:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.497668:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986113 is committed 00002000:00010000:0.0:1789760779.497994:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77029376 -> 77033472 00002000:00010000:0.0:1789760779.497999:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150472 -> 150480 00010000:00010000:0.0:1789760779.498013:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498026:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498035:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498043:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.498194:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498206:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498226:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498234:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0a43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77029376->77033471] (req 77029376->77033471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.498478:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7729us (8014us total) trans 4294986113 rc 0/0 00000100:00100000:0.0:1789760779.498489:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18817 00000100:00100000:0.0:1789760779.498499:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709597843533 time 709615338102 state 0x2 ref 0 00000100:00100000:1.0:1789760779.502066:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.502161:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795584 00000100:00100000:3.0:1789760779.502172:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795584 00000100:00100000:3.0:1789760779.502225:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709597843533 time 709615338102 state 0x0 ref 1 00000100:00100000:3.0:1789760779.502240:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709615598518 time 709615338102 state 0x1 ref 1 00000100:00100000:3.0:1789760779.502272:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709615598518 time 709626878319 state 0x0 ref 1 00000100:00100000:3.0:1789760779.502284:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.502498:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77033472, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bd0 00010000:00010000:3.0:1789760779.502508:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.502523:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.502528:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.502534:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.502547:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.510126:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986114 is committed 00002000:00010000:3.0:1789760779.510306:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77033472 -> 77037568 00002000:00010000:3.0:1789760779.510311:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150480 -> 150488 00010000:00010000:3.0:1789760779.510325:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.510339:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.510348:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.510355:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.510360:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.510370:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.510473:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.510481:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77033472->77037567] (req 77033472->77037567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.510712:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8424us (8649us total) trans 4294986114 rc 0/0 00000100:00100000:3.0:1789760779.510723:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18818 00000100:00100000:3.0:1789760779.510733:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709615598518 time 709626878319 state 0x2 ref 0 00000100:00100000:3.0:1789760779.516108:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.516170:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795712 00000100:00100000:1.0:1789760779.516181:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795712 00000100:00100000:1.0:1789760779.516234:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709615598518 time 709626878319 state 0x0 ref 1 00000100:00100000:1.0:1789760779.516248:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709627138735 time 709626878319 state 0x1 ref 1 00000100:00100000:1.0:1789760779.516279:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709627138735 time 709640886095 state 0x0 ref 1 00000100:00100000:1.0:1789760779.516291:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc14c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.516337:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77037568, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bd8 00010000:00010000:1.0:1789760779.516346:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.516364:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.516369:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.516375:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.516419:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.521928:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986115 is committed 00002000:00010000:3.0:1789760779.522213:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77037568 -> 77041664 00002000:00010000:3.0:1789760779.522220:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150488 -> 150496 00010000:00010000:3.0:1789760779.522233:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.522733:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.522743:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.522751:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.522756:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.522766:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.522786:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.522795:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f0a51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77037568->77041663] (req 77037568->77041663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.522928:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc14c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6633us (6826us total) trans 4294986115 rc 0/0 00000100:00100000:3.0:1789760779.522939:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18819 00000100:00100000:3.0:1789760779.522948:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709627138735 time 709640886095 state 0x2 ref 0 00000100:00100000:0.0:1789760779.527684:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.527777:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795840 00000100:00100000:2.0:1789760779.527788:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795840 00000100:00100000:2.0:1789760779.527843:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709627138735 time 709640886095 state 0x0 ref 1 00000100:00100000:2.0:1789760779.527858:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709641146511 time 709640886095 state 0x1 ref 1 00000100:00100000:2.0:1789760779.527920:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709641146511 time 709652526539 state 0x0 ref 1 00000100:00100000:2.0:1789760779.527934:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.527985:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77041664, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24be0 00010000:00010000:2.0:1789760779.527996:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.528013:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.528018:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.528024:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.528038:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.533706:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986116 is committed 00002000:00010000:0.0:1789760779.533960:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77041664 -> 77045760 00002000:00010000:0.0:1789760779.533965:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150496 -> 150504 00010000:00010000:0.0:1789760779.533979:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.533991:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.533999:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.534006:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.534010:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.534017:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.534034:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.534042:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0a58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77041664->77045759] (req 77041664->77045759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.534130:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503795840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6194us (6452us total) trans 4294986116 rc 0/0 00000100:00100000:0.0:1789760779.534140:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18820 00000100:00100000:0.0:1789760779.534150:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709641146511 time 709652526539 state 0x2 ref 0 00000100:00100000:1.0:1789760779.537111:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.537250:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503795968 00000100:00100000:0.0:1789760779.537260:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503795968 00000100:00100000:0.0:1789760779.537310:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709641146511 time 709652526539 state 0x0 ref 1 00000100:00100000:0.0:1789760779.537326:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709652786955 time 709652526539 state 0x1 ref 1 00000100:00100000:0.0:1789760779.537356:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709652786955 time 709661962844 state 0x0 ref 1 00000100:00100000:0.0:1789760779.537369:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.537477:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77045760, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24be8 00010000:00010000:0.0:1789760779.537486:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.537500:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.537504:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.537511:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.537527:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.542863:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986117 is committed 00002000:00010000:0.0:1789760779.543168:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77045760 -> 77049856 00002000:00010000:0.0:1789760779.543174:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150504 -> 150512 00010000:00010000:0.0:1789760779.543188:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.543202:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.543211:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.543218:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.543223:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.543232:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.543252:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.543260:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0a5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77045760->77049855] (req 77045760->77049855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.543495:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503795968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6122us (6387us total) trans 4294986117 rc 0/0 00000100:00100000:0.0:1789760779.543506:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18821 00000100:00100000:0.0:1789760779.543517:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709652786955 time 709661962844 state 0x2 ref 0 00000100:00100000:3.0:1789760779.546127:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.546204:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796096 00000100:00100000:0.0:1789760779.546213:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796096 00000100:00100000:0.0:1789760779.546261:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709652786955 time 709661962844 state 0x0 ref 1 00000100:00100000:0.0:1789760779.546275:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709662223260 time 709661962844 state 0x1 ref 1 00000100:00100000:0.0:1789760779.546305:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709662223260 time 709670912460 state 0x0 ref 1 00000100:00100000:0.0:1789760779.546323:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.546373:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77049856, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bf0 00010000:00010000:0.0:1789760779.546381:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.546437:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.546445:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.546452:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.546469:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.551001:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986118 is committed 00002000:00010000:0.0:1789760779.551155:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77049856 -> 77053952 00002000:00010000:0.0:1789760779.551161:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150512 -> 150520 00010000:00010000:0.0:1789760779.551175:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.551188:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.551198:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.551205:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.551210:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.551219:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.551239:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.551246:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0a66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77049856->77053951] (req 77049856->77053951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.551347:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5021us (5222us total) trans 4294986118 rc 0/0 00000100:00100000:0.0:1789760779.551357:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18822 00000100:00100000:0.0:1789760779.551367:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709662223260 time 709670912460 state 0x2 ref 0 00000100:00100000:2.0:1789760779.554358:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.554535:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796224 00000100:00100000:3.0:1789760779.554545:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796224 00000100:00100000:3.0:1789760779.554608:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709662223260 time 709670912460 state 0x0 ref 1 00000100:00100000:3.0:1789760779.554632:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709671172876 time 709670912460 state 0x1 ref 1 00000100:00100000:3.0:1789760779.554692:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709671172876 time 709679298301 state 0x0 ref 1 00000100:00100000:3.0:1789760779.554705:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.554750:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77053952, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24bf8 00010000:00010000:3.0:1789760779.554760:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.554776:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.554780:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.554787:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.554801:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.558721:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986119 is committed 00002000:00010000:3.0:1789760779.559251:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77053952 -> 77058048 00002000:00010000:3.0:1789760779.559258:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150520 -> 150528 00010000:00010000:3.0:1789760779.559271:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.559284:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.559294:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.559301:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.559439:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.559452:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.559477:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.559489:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77053952->77058047] (req 77053952->77058047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.559655:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4946us (5301us total) trans 4294986119 rc 0/0 00000100:00100000:3.0:1789760779.559666:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18823 00000100:00100000:3.0:1789760779.559685:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709671172876 time 709679298301 state 0x2 ref 0 00000100:00100000:1.0:1789760779.562427:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.562539:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796352 00000100:00100000:2.0:1789760779.562549:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796352 00000100:00100000:2.0:1789760779.562613:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709671172876 time 709679298301 state 0x0 ref 1 00000100:00100000:2.0:1789760779.562629:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709679558717 time 709679298301 state 0x1 ref 1 00000100:00100000:2.0:1789760779.562649:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709679558717 time 709687256872 state 0x0 ref 1 00000100:00100000:2.0:1789760779.562662:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503796352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.562708:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77058048, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c00 00010000:00010000:2.0:1789760779.562718:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.562739:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.562743:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.562750:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.562764:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.566359:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986120 is committed 00002000:00010000:2.0:1789760779.566682:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77058048 -> 77062144 00002000:00010000:2.0:1789760779.566687:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150528 -> 150536 00010000:00010000:2.0:1789760779.566700:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.566713:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.566723:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.566731:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.566736:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.566745:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.566772:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.566780:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77058048->77062143] (req 77058048->77062143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.566989:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503796352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4323us (4605us total) trans 4294986120 rc 0/0 00000100:00100000:2.0:1789760779.567000:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18824 00000100:00100000:2.0:1789760779.567010:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709679558717 time 709687256872 state 0x2 ref 0 00000100:00100000:0.0:1789760779.570544:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.570843:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796480 00000100:00100000:2.0:1789760779.570853:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796480 00000100:00100000:2.0:1789760779.570969:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709679558717 time 709687256872 state 0x0 ref 1 00000100:00100000:2.0:1789760779.570985:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709687517288 time 709687256872 state 0x1 ref 1 00000100:00100000:2.0:1789760779.571072:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709687517288 time 709695677433 state 0x0 ref 1 00000100:00100000:2.0:1789760779.571086:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.571134:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77062144, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c08 00010000:00010000:2.0:1789760779.571143:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.571158:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.571163:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.571169:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.571183:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.575134:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986121 is committed 00002000:00010000:2.0:1789760779.575525:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77062144 -> 77066240 00002000:00010000:2.0:1789760779.575531:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150536 -> 150544 00010000:00010000:2.0:1789760779.575544:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.575555:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.575563:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.575570:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.575574:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.575593:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.575615:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.575628:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77062144->77066239] (req 77062144->77066239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.575846:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4756us (5305us total) trans 4294986121 rc 0/0 00000100:00100000:2.0:1789760779.575858:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18825 00000100:00100000:2.0:1789760779.575868:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709687517288 time 709695677433 state 0x2 ref 0 00000100:00100000:0.0:1789760779.579509:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.579610:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796608 00000100:00100000:3.0:1789760779.579687:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796608 00000100:00100000:3.0:1789760779.579747:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709687517288 time 709695677433 state 0x0 ref 1 00000100:00100000:3.0:1789760779.579761:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709695937849 time 709695677433 state 0x1 ref 1 00000100:00100000:3.0:1789760779.579826:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709695937849 time 709704431239 state 0x0 ref 1 00000100:00100000:3.0:1789760779.579838:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503796608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.579887:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77066240, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c10 00010000:00010000:3.0:1789760779.579896:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.579914:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.579919:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.579925:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.579938:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.584457:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986122 is committed 00002000:00010000:3.0:1789760779.585196:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77066240 -> 77070336 00002000:00010000:3.0:1789760779.585203:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150544 -> 150552 00010000:00010000:3.0:1789760779.585221:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.585237:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.585247:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.585256:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.585261:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.585270:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.585292:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.585300:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77066240->77070335] (req 77066240->77070335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.585558:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503796608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5716us (6064us total) trans 4294986122 rc 0/0 00000100:00100000:3.0:1789760779.585569:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18826 00000100:00100000:3.0:1789760779.585593:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709695937849 time 709704431239 state 0x2 ref 0 00000100:00100000:1.0:1789760779.588194:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.588559:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796736 00000100:00100000:3.0:1789760779.588569:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796736 00000100:00100000:2.0:1789760779.588771:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709695937849 time 709704431239 state 0x0 ref 1 00000100:00100000:2.0:1789760779.588788:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709704691655 time 709704431239 state 0x1 ref 1 00000100:00100000:2.0:1789760779.588861:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709704691655 time 709713466177 state 0x0 ref 1 00000100:00100000:2.0:1789760779.588875:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.588922:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77070336, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c18 00010000:00010000:2.0:1789760779.588931:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.588946:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.588951:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.588958:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.588971:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.592215:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986123 is committed 00002000:00010000:2.0:1789760779.592711:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77070336 -> 77074432 00002000:00010000:2.0:1789760779.592717:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150552 -> 150560 00010000:00010000:2.0:1789760779.592730:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.592742:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.592751:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.592758:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.592763:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.592772:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.592791:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.592798:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77070336->77074431] (req 77070336->77074431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.592931:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4053us (4741us total) trans 4294986123 rc 0/0 00000100:00100000:2.0:1789760779.592942:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18827 00000100:00100000:2.0:1789760779.592952:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709704691655 time 709713466177 state 0x2 ref 0 00000100:00100000:0.0:1789760779.597545:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.597600:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796864 00000100:00100000:2.0:1789760779.597611:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796864 00000100:00100000:2.0:1789760779.597670:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709704691655 time 709713466177 state 0x0 ref 1 00000100:00100000:2.0:1789760779.597685:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709713726593 time 709713466177 state 0x1 ref 1 00000100:00100000:2.0:1789760779.597715:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709713726593 time 709722321422 state 0x0 ref 1 00000100:00100000:2.0:1789760779.597726:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.597775:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77074432, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c20 00010000:00010000:2.0:1789760779.597784:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.597802:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.597807:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.597814:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.597828:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.601099:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986124 is committed 00002000:00010000:2.0:1789760779.601240:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77074432 -> 77078528 00002000:00010000:2.0:1789760779.601246:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150560 -> 150568 00010000:00010000:2.0:1789760779.601259:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.601272:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.601281:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.601289:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.601295:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.601304:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.601876:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.601884:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0a90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77074432->77078527] (req 77074432->77078527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.602154:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4423us (4612us total) trans 4294986124 rc 0/0 00000100:00100000:2.0:1789760779.602165:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18828 00000100:00100000:2.0:1789760779.602174:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709713726593 time 709722321422 state 0x2 ref 0 00000100:00100000:0.0:1789760779.605652:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.605847:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503796992 00000100:00100000:2.0:1789760779.605858:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503796992 00000100:00100000:2.0:1789760779.605915:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709713726593 time 709722321422 state 0x0 ref 1 00000100:00100000:2.0:1789760779.605930:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709722581838 time 709722321422 state 0x1 ref 1 00000100:00100000:2.0:1789760779.605962:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709722581838 time 709730568578 state 0x0 ref 1 00000100:00100000:2.0:1789760779.605974:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.606021:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77078528, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c28 00010000:00010000:2.0:1789760779.606031:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.606047:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.606051:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.606058:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.606071:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.610223:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986125 is committed 00002000:00010000:2.0:1789760779.610531:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77078528 -> 77082624 00002000:00010000:2.0:1789760779.610536:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150568 -> 150576 00010000:00010000:2.0:1789760779.610549:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.610567:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.610575:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.610592:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.610597:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.610606:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.610640:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.610649:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0a97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77078528->77082623] (req 77078528->77082623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.610774:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503796992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4795us (5124us total) trans 4294986125 rc 0/0 00000100:00100000:2.0:1789760779.610784:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18829 00000100:00100000:2.0:1789760779.610794:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709722581838 time 709730568578 state 0x2 ref 0 00000100:00100000:1.0:1789760779.614795:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.615377:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503797120 00000100:00100000:3.0:1789760779.615485:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503797120 00000100:00100000:3.0:1789760779.615568:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709722581838 time 709730568578 state 0x0 ref 1 00000100:00100000:3.0:1789760779.615596:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709730828994 time 709730568578 state 0x1 ref 1 00000100:00100000:3.0:1789760779.615723:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709730828994 time 709740327916 state 0x0 ref 1 00000100:00100000:3.0:1789760779.615736:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.615787:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77082624, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c30 00010000:00010000:3.0:1789760779.615797:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.615814:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760779.615818:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.615824:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.615839:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.620101:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986126 is committed 00002000:00010000:3.0:1789760779.620230:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77082624 -> 77086720 00002000:00010000:3.0:1789760779.620237:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150576 -> 150584 00010000:00010000:3.0:1789760779.620253:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.620269:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.620280:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.620289:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.620295:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.620305:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.620328:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.620337:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0a9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77082624->77086719] (req 77082624->77086719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.620603:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4865us (5813us total) trans 4294986126 rc 0/0 00000100:00100000:3.0:1789760779.620614:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18830 00000100:00100000:3.0:1789760779.620623:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709730828994 time 709740327916 state 0x2 ref 0 00000100:00100000:0.0:1789760779.624691:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.624914:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503797248 00000100:00100000:2.0:1789760779.624925:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503797248 00000100:00100000:2.0:1789760779.624982:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709730828994 time 709740327916 state 0x0 ref 1 00000100:00100000:2.0:1789760779.624997:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709740588332 time 709740327916 state 0x1 ref 1 00000100:00100000:2.0:1789760779.625028:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709740588332 time 709749634671 state 0x0 ref 1 00000100:00100000:2.0:1789760779.625043:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503797248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.625091:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77086720, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c38 00010000:00010000:2.0:1789760779.625101:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.625117:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760779.625122:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.625128:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.625141:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.629497:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986127 is committed 00002000:00010000:1.0:1789760779.629630:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77086720 -> 77090816 00002000:00010000:1.0:1789760779.629635:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150584 -> 150592 00010000:00010000:1.0:1789760779.629648:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.629661:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.629678:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.629687:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.629692:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.629702:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.629725:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.629738:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0aa5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77086720->77090815] (req 77086720->77090815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.629916:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503797248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4870us (5229us total) trans 4294986127 rc 0/0 00000100:00100000:1.0:1789760779.629927:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18831 00000100:00100000:1.0:1789760779.629938:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709740588332 time 709749634671 state 0x2 ref 0 00000100:00100000:2.0:1789760779.633331:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.633749:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503797376 00000100:00100000:0.0:1789760779.633759:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503797376 00000100:00100000:0.0:1789760779.633817:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709740588332 time 709749634671 state 0x0 ref 1 00000100:00100000:0.0:1789760779.633832:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709749895087 time 709749634671 state 0x1 ref 1 00000100:00100000:0.0:1789760779.633867:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709749895087 time 709758473544 state 0x0 ref 1 00000100:00100000:0.0:1789760779.633879:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.633926:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77090816, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c40 00010000:00010000:0.0:1789760779.633934:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.633949:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.633954:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.633960:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.633974:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.638088:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986128 is committed 00002000:00010000:0.0:1789760779.638181:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77090816 -> 77094912 00002000:00010000:0.0:1789760779.638186:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150592 -> 150600 00010000:00010000:0.0:1789760779.638199:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638213:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638222:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638229:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.638239:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638249:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638268:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638276:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77090816->77094911] (req 77090816->77094911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.639099:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5215us (5770us total) trans 4294986128 rc 0/0 00000100:00100000:0.0:1789760779.639114:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18832 00000100:00100000:0.0:1789760779.639123:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709749895087 time 709758473544 state 0x2 ref 0 00000100:00100000:2.0:1789760779.643303:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.643470:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503797504 00000100:00100000:0.0:1789760779.643479:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503797504 00000100:00100000:0.0:1789760779.643533:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709749895087 time 709758473544 state 0x0 ref 1 00000100:00100000:0.0:1789760779.643548:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709758733960 time 709758473544 state 0x1 ref 1 00000100:00100000:0.0:1789760779.643624:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709758733960 time 709768229186 state 0x0 ref 1 00000100:00100000:0.0:1789760779.643637:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503797504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.643754:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77094912, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c48 00010000:00010000:0.0:1789760779.643764:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.643780:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.643785:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.643792:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.643806:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.648523:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986129 is committed 00002000:00010000:0.0:1789760779.651262:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77094912 -> 77099008 00002000:00010000:0.0:1789760779.651273:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150600 -> 150608 00010000:00010000:0.0:1789760779.651288:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.651301:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.651311:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.651319:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.651323:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.651333:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.651354:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.651370:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ab3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77094912->77099007] (req 77094912->77099007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.651505:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503797504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7865us (8207us total) trans 4294986129 rc 0/0 00000100:00100000:0.0:1789760779.651516:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18833 00000100:00100000:0.0:1789760779.651526:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709758733960 time 709768229186 state 0x2 ref 0 00000100:00100000:2.0:1789760779.654535:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.654653:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503797632 00000100:00100000:1.0:1789760779.654663:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503797632 00000100:00100000:1.0:1789760779.654719:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709758733960 time 709768229186 state 0x0 ref 1 00000100:00100000:1.0:1789760779.654736:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709768489602 time 709768229186 state 0x1 ref 1 00000100:00100000:1.0:1789760779.654767:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709768489602 time 709779373358 state 0x0 ref 1 00000100:00100000:1.0:1789760779.654780:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.654828:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77099008, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c50 00010000:00010000:1.0:1789760779.654837:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.654851:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.654856:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.654862:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.654881:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.659911:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986130 is committed 00002000:00010000:1.0:1789760779.660032:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77099008 -> 77103104 00002000:00010000:1.0:1789760779.660039:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150608 -> 150616 00010000:00010000:1.0:1789760779.660053:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.660068:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.660079:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.660087:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.660092:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.660103:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.660125:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.660133:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0aba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77099008->77103103] (req 77099008->77103103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.660288:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5504us (5757us total) trans 4294986130 rc 0/0 00000100:00100000:1.0:1789760779.660301:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18834 00000100:00100000:1.0:1789760779.660325:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709768489602 time 709779373358 state 0x2 ref 0 00000100:00100000:2.0:1789760779.665498:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.665645:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503797760 00000100:00100000:1.0:1789760779.665656:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503797760 00000100:00100000:1.0:1789760779.665709:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709768489602 time 709779373358 state 0x0 ref 1 00000100:00100000:1.0:1789760779.665724:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709779633774 time 709779373358 state 0x1 ref 1 00000100:00100000:1.0:1789760779.665754:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709779633774 time 709790360574 state 0x0 ref 1 00000100:00100000:1.0:1789760779.665765:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503797760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.665812:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77103104, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c58 00010000:00010000:1.0:1789760779.665822:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.665836:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.665841:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.665847:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.665861:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.672029:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986131 is committed 00002000:00010000:1.0:1789760779.672185:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77103104 -> 77107200 00002000:00010000:1.0:1789760779.672190:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150616 -> 150624 00010000:00010000:1.0:1789760779.672203:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.672216:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.672225:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.672233:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.672237:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.672246:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.672261:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.672266:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0ac1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77103104->77107199] (req 77103104->77107199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.672355:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503797760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6586us (6862us total) trans 4294986131 rc 0/0 00000100:00100000:1.0:1789760779.672367:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18835 00000100:00100000:1.0:1789760779.672382:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709779633774 time 709790360574 state 0x2 ref 0 00000100:00100000:2.0:1789760779.675966:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.676046:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503797888 00000100:00100000:0.0:1789760779.676055:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503797888 00000100:00100000:0.0:1789760779.676106:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709779633774 time 709790360574 state 0x0 ref 1 00000100:00100000:0.0:1789760779.676121:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709790620990 time 709790360574 state 0x1 ref 1 00000100:00100000:0.0:1789760779.676184:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709790620990 time 709800790515 state 0x0 ref 1 00000100:00100000:0.0:1789760779.676196:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.676248:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77107200, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c60 00010000:00010000:0.0:1789760779.676257:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.676272:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.676277:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.676283:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.676297:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.681463:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986132 is committed 00002000:00010000:0.0:1789760779.681740:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77107200 -> 77111296 00002000:00010000:0.0:1789760779.681746:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150624 -> 150632 00010000:00010000:0.0:1789760779.681760:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.681774:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.681783:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.681791:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.681797:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.681806:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.681827:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.681835:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ac8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77107200->77111295] (req 77107200->77111295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.681969:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503797888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5770us (6007us total) trans 4294986132 rc 0/0 00000100:00100000:0.0:1789760779.681980:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18836 00000100:00100000:0.0:1789760779.681991:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709790620990 time 709800790515 state 0x2 ref 0 00000100:00100000:2.0:1789760779.688844:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.688941:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798016 00000100:00100000:1.0:1789760779.688952:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798016 00000100:00100000:1.0:1789760779.689007:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709790620990 time 709800790515 state 0x0 ref 1 00000100:00100000:1.0:1789760779.689023:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709801050931 time 709800790515 state 0x1 ref 1 00000100:00100000:1.0:1789760779.689081:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709801050931 time 709813687635 state 0x0 ref 1 00000100:00100000:1.0:1789760779.689104:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.689152:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77111296, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c68 00010000:00010000:1.0:1789760779.689162:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.689178:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.689182:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.689189:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.689204:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.693386:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986133 is committed 00002000:00010000:1.0:1789760779.693721:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77111296 -> 77115392 00002000:00010000:1.0:1789760779.693727:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150632 -> 150640 00010000:00010000:1.0:1789760779.693741:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.693753:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.693761:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.693767:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.693772:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.693780:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.693798:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.693807:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0acf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77111296->77115391] (req 77111296->77115391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.693903:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4805us (5065us total) trans 4294986133 rc 0/0 00000100:00100000:1.0:1789760779.693913:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18837 00000100:00100000:1.0:1789760779.693922:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709801050931 time 709813687635 state 0x2 ref 0 00000100:00100000:2.0:1789760779.698155:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.698247:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798144 00000100:00100000:1.0:1789760779.698257:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798144 00000100:00100000:1.0:1789760779.698751:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709801050931 time 709813687635 state 0x0 ref 1 00000100:00100000:1.0:1789760779.698771:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709813948051 time 709813687635 state 0x1 ref 1 00000100:00100000:1.0:1789760779.698842:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709813948051 time 709823447651 state 0x0 ref 1 00000100:00100000:1.0:1789760779.698856:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503798144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.698904:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77115392, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c70 00010000:00010000:1.0:1789760779.698913:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.698929:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.698933:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.698940:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.698953:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.704446:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986134 is committed 00002000:00010000:1.0:1789760779.704850:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77115392 -> 77119488 00002000:00010000:1.0:1789760779.704856:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150640 -> 150648 00010000:00010000:1.0:1789760779.704869:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.704888:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.704897:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.704904:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.704909:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.704918:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.704937:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.704945:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0ad6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77115392->77119487] (req 77115392->77119487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.705109:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503798144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6251us (6956us total) trans 4294986134 rc 0/0 00000100:00100000:1.0:1789760779.705119:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18838 00000100:00100000:1.0:1789760779.705130:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709813948051 time 709823447651 state 0x2 ref 0 00000100:00100000:2.0:1789760779.710779:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.710881:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798272 00000100:00100000:1.0:1789760779.710892:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798272 00000100:00100000:1.0:1789760779.710956:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709813948051 time 709823447651 state 0x0 ref 1 00000100:00100000:1.0:1789760779.710972:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709823708067 time 709823447651 state 0x1 ref 1 00000100:00100000:1.0:1789760779.711052:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709823708067 time 709835656657 state 0x0 ref 1 00000100:00100000:1.0:1789760779.711065:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.711114:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77119488, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c78 00010000:00010000:1.0:1789760779.711124:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.711138:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.711142:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.711147:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.711160:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.714968:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986135 is committed 00002000:00010000:1.0:1789760779.715062:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77119488 -> 77123584 00002000:00010000:1.0:1789760779.715067:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150648 -> 150656 00010000:00010000:1.0:1789760779.715080:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.715092:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.715107:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.715115:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.715119:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.715128:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.715148:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.715155:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0add lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77119488->77123583] (req 77119488->77123583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.715285:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4218us (4510us total) trans 4294986135 rc 0/0 00000100:00100000:1.0:1789760779.715296:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18839 00000100:00100000:1.0:1789760779.715306:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709823708067 time 709835656657 state 0x2 ref 0 00000100:00100000:2.0:1789760779.719710:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.722822:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798400 00000100:00100000:0.0:1789760779.722838:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798400 00000100:00100000:0.0:1789760779.722904:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709823708067 time 709835656657 state 0x0 ref 1 00000100:00100000:0.0:1789760779.722921:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709835917073 time 709835656657 state 0x1 ref 1 00000100:00100000:0.0:1789760779.723055:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709835917073 time 709847660079 state 0x0 ref 1 00000100:00100000:0.0:1789760779.723070:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503798400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.723118:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77123584, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c80 00010000:00010000:0.0:1789760779.723128:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.723144:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.723149:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.723155:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.723169:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.732479:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986136 is committed 00002000:00010000:0.0:1789760779.732624:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77123584 -> 77127680 00002000:00010000:0.0:1789760779.732630:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150656 -> 150664 00010000:00010000:0.0:1789760779.732644:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.732663:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.732671:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.732678:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.732683:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.732693:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.732716:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.732723:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0ae4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77123584->77127679] (req 77123584->77127679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.732814:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503798400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9743us (13107us total) trans 4294986136 rc 0/0 00000100:00100000:0.0:1789760779.732824:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18840 00000100:00100000:0.0:1789760779.732833:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709835917073 time 709847660079 state 0x2 ref 0 00000100:00100000:2.0:1789760779.740882:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.741064:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798528 00000100:00100000:0.0:1789760779.741076:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798528 00000100:00100000:0.0:1789760779.741139:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709835917073 time 709847660079 state 0x0 ref 1 00000100:00100000:0.0:1789760779.741155:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709847920495 time 709847660079 state 0x1 ref 1 00000100:00100000:0.0:1789760779.741187:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709847920495 time 709865793701 state 0x0 ref 1 00000100:00100000:0.0:1789760779.741200:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503798528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.741250:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77127680, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c88 00010000:00010000:0.0:1789760779.741260:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.741275:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.741280:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.741287:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.741300:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.746675:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986137 is committed 00002000:00010000:0.0:1789760779.746841:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77127680 -> 77131776 00002000:00010000:0.0:1789760779.746847:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150664 -> 150672 00010000:00010000:0.0:1789760779.746859:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.746871:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.746885:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.746892:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.746897:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.746905:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.746925:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.746933:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0aeb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77127680->77131775] (req 77127680->77131775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.747034:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503798528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5831us (6156us total) trans 4294986137 rc 0/0 00000100:00100000:0.0:1789760779.747044:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18841 00000100:00100000:0.0:1789760779.747054:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709847920495 time 709865793701 state 0x2 ref 0 00000100:00100000:2.0:1789760779.750768:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.750916:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798656 00000100:00100000:1.0:1789760779.750926:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798656 00000100:00100000:1.0:1789760779.750979:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709847920495 time 709865793701 state 0x0 ref 1 00000100:00100000:1.0:1789760779.751259:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709866054117 time 709865793701 state 0x1 ref 1 00000100:00100000:1.0:1789760779.751293:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709866054117 time 709875899236 state 0x0 ref 1 00000100:00100000:1.0:1789760779.751312:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.751358:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77131776, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c90 00010000:00010000:1.0:1789760779.751367:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751382:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.751414:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.751421:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.751436:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.765799:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986138 is committed 00002000:00010000:1.0:1789760779.766928:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77131776 -> 77135872 00002000:00010000:1.0:1789760779.766937:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150672 -> 150680 00010000:00010000:1.0:1789760779.767027:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767040:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767189:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767203:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.767208:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767218:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767242:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767249:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77131776->77135871] (req 77131776->77135871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.767358:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16047us (16594us total) trans 4294986138 rc 0/0 00000100:00100000:1.0:1789760779.767370:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18842 00000100:00100000:1.0:1789760779.767382:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709866054117 time 709875899236 state 0x2 ref 0 00000100:00100000:2.0:1789760779.770912:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.771972:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798784 00000100:00100000:1.0:1789760779.771986:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798784 00000100:00100000:1.0:1789760779.772524:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709866054117 time 709875899236 state 0x0 ref 1 00000100:00100000:1.0:1789760779.772547:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709876159652 time 709875899236 state 0x1 ref 1 00000100:00100000:1.0:1789760779.772580:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709876159652 time 709897186450 state 0x0 ref 1 00000100:00100000:1.0:1789760779.772693:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503798784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.772740:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77135872, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24c98 00010000:00010000:1.0:1789760779.772749:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.772765:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.772769:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.772775:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.772788:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.780528:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986139 is committed 00002000:00010000:0.0:1789760779.780658:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77135872 -> 77139968 00002000:00010000:0.0:1789760779.780663:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150680 -> 150688 00010000:00010000:0.0:1789760779.780676:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.780692:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.780702:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.780708:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.780712:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.780721:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.780740:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.780747:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0af9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77135872->77139967] (req 77135872->77139967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.780849:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503798784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8157us (9941us total) trans 4294986139 rc 0/0 00000100:00100000:0.0:1789760779.780859:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18843 00000100:00100000:0.0:1789760779.780873:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709876159652 time 709897186450 state 0x2 ref 0 00000100:00100000:2.0:1789760779.784245:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.784372:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503798912 00000100:00100000:0.0:1789760779.784384:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503798912 00000100:00100000:0.0:1789760779.784534:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709876159652 time 709897186450 state 0x0 ref 1 00000100:00100000:0.0:1789760779.784548:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709897446866 time 709897186450 state 0x1 ref 1 00000100:00100000:0.0:1789760779.784569:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709897446866 time 709909176845 state 0x0 ref 1 00000100:00100000:0.0:1789760779.784676:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.784723:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77139968, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x24ca0 00010000:00010000:0.0:1789760779.784730:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.784743:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.784747:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.784751:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.784761:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.790306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986140 is committed 00002000:00010000:0.0:1789760779.794254:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77139968 -> 77144064 00002000:00010000:0.0:1789760779.794263:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760779.794265:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760779.794267:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150688 -> 150696 00010000:00010000:0.0:1789760779.794281:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.794300:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.794309:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.794316:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.794321:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.794330:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.794350:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.794357:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77139968->77144063] (req 77139968->77144063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.795708:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503798912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11119us (11464us total) trans 4294986140 rc 0/0 00000100:00100000:0.0:1789760779.795722:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18844 00000100:00100000:0.0:1789760779.795732:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709897446866 time 709909176845 state 0x2 ref 0 00010000:00010000:3.0:1789760779.796854:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760779.798909:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.799024:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799040 00000100:00100000:0.0:1789760779.799035:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799040 00000100:00100000:0.0:1789760779.799163:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709897446866 time 709909176845 state 0x0 ref 1 00000100:00100000:0.0:1789760779.799182:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709909437261 time 709909176845 state 0x1 ref 1 00000100:00100000:0.0:1789760779.799216:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709909437261 time 709923821993 state 0x0 ref 1 00000100:00100000:0.0:1789760779.799228:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.799275:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77144064, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ca8 00010000:00010000:0.0:1789760779.799285:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.799300:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.799305:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.799311:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.799325:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.805019:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986141 is committed 00002000:00010000:0.0:1789760779.805508:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77144064 -> 77148160 00002000:00010000:0.0:1789760779.805513:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150696 -> 150704 00010000:00010000:0.0:1789760779.805527:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.805539:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.805553:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.805560:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.805565:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.805573:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.805602:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.805610:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77144064->77148159] (req 77144064->77148159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.805713:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6482us (6809us total) trans 4294986141 rc 0/0 00000100:00100000:0.0:1789760779.805724:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18845 00000100:00100000:0.0:1789760779.805733:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709909437261 time 709923821993 state 0x2 ref 0 00000100:00100000:2.0:1789760779.809709:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.809777:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799168 00000100:00100000:1.0:1789760779.809787:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799168 00000100:00100000:1.0:1789760779.809837:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709909437261 time 709923821993 state 0x0 ref 1 00000100:00100000:1.0:1789760779.809851:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709924082409 time 709923821993 state 0x1 ref 1 00000100:00100000:1.0:1789760779.809880:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709924082409 time 709934487418 state 0x0 ref 1 00000100:00100000:1.0:1789760779.809892:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.809940:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77148160, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cb0 00010000:00010000:1.0:1789760779.809950:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.809969:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.809974:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.809980:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.809994:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.817851:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986142 is committed 00002000:00010000:1.0:1789760779.817977:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77148160 -> 77152256 00002000:00010000:1.0:1789760779.817982:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150704 -> 150712 00010000:00010000:1.0:1789760779.817994:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.818006:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.818014:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.818022:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.818027:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.818035:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.818061:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.818068:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77148160->77152255] (req 77148160->77152255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.818161:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8266us (8457us total) trans 4294986142 rc 0/0 00000100:00100000:1.0:1789760779.818172:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18846 00000100:00100000:1.0:1789760779.818180:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709924082409 time 709934487418 state 0x2 ref 0 00000100:00100000:2.0:1789760779.825508:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.826431:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799296 00000100:00100000:0.0:1789760779.826442:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799296 00000100:00100000:0.0:1789760779.826500:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709924082409 time 709934487418 state 0x0 ref 1 00000100:00100000:0.0:1789760779.826518:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709934747834 time 709934487418 state 0x1 ref 1 00000100:00100000:0.0:1789760779.826554:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709934747834 time 709951161167 state 0x0 ref 1 00000100:00100000:0.0:1789760779.826567:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.826627:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77152256, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cb8 00010000:00010000:0.0:1789760779.826636:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.826651:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.826655:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.826661:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.826675:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.834898:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986143 is committed 00002000:00010000:2.0:1789760779.835207:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77152256 -> 77156352 00002000:00010000:2.0:1789760779.835213:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150712 -> 150720 00010000:00010000:2.0:1789760779.835227:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.835241:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.835250:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.835262:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.835267:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.835275:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.835294:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.835302:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0b15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77152256->77156351] (req 77152256->77156351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.835464:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8894us (9960us total) trans 4294986143 rc 0/0 00000100:00100000:2.0:1789760779.835475:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18847 00000100:00100000:2.0:1789760779.835486:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709934747834 time 709951161167 state 0x2 ref 0 00000100:00100000:2.0:1789760779.839369:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.839546:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799424 00000100:00100000:0.0:1789760779.839556:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799424 00000100:00100000:0.0:1789760779.839622:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709934747834 time 709951161167 state 0x0 ref 1 00000100:00100000:0.0:1789760779.839635:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709951421583 time 709951161167 state 0x1 ref 1 00000100:00100000:0.0:1789760779.839665:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709951421583 time 709964272495 state 0x0 ref 1 00000100:00100000:0.0:1789760779.839680:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.839722:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77156352, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cc0 00010000:00010000:0.0:1789760779.839731:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.839746:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760779.839750:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.839755:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.839767:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.848459:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986144 is committed 00002000:00010000:0.0:1789760779.848593:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77156352 -> 77160448 00002000:00010000:0.0:1789760779.848599:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150720 -> 150728 00010000:00010000:0.0:1789760779.848613:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.848626:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.848635:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.848714:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760779.848719:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.848729:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.848748:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.848756:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77156352->77160447] (req 77156352->77160447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.848885:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9201us (9520us total) trans 4294986144 rc 0/0 00000100:00100000:0.0:1789760779.848896:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18848 00000100:00100000:0.0:1789760779.848906:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709951421583 time 709964272495 state 0x2 ref 0 00000100:00100000:0.0:1789760779.855716:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.856071:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799552 00000100:00100000:1.0:1789760779.856082:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799552 00000100:00100000:1.0:1789760779.856202:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709951421583 time 709964272495 state 0x0 ref 1 00000100:00100000:1.0:1789760779.856218:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709964532911 time 709964272495 state 0x1 ref 1 00000100:00100000:1.0:1789760779.856251:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709964532911 time 709980857190 state 0x0 ref 1 00000100:00100000:1.0:1789760779.856264:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.856410:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77160448, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cc8 00010000:00010000:1.0:1789760779.856421:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.856438:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.856443:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.856449:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.856465:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.866082:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986145 is committed 00002000:00010000:1.0:1789760779.866757:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77160448 -> 77164544 00002000:00010000:1.0:1789760779.866763:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150728 -> 150736 00010000:00010000:1.0:1789760779.866841:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.866855:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.866864:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.866872:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.866877:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.866886:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.866906:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.866919:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77160448->77164543] (req 77160448->77164543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.867079:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10811us (11366us total) trans 4294986145 rc 0/0 00000100:00100000:1.0:1789760779.867099:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18849 00000100:00100000:1.0:1789760779.867110:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709964532911 time 709980857190 state 0x2 ref 0 00000100:00100000:2.0:1789760779.872681:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.872851:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799680 00000100:00100000:1.0:1789760779.872861:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799680 00000100:00100000:1.0:1789760779.872917:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709964532911 time 709980857190 state 0x0 ref 1 00000100:00100000:1.0:1789760779.872932:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709981117606 time 709980857190 state 0x1 ref 1 00000100:00100000:1.0:1789760779.873004:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709981117606 time 709997610584 state 0x0 ref 1 00000100:00100000:1.0:1789760779.873018:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.873357:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77164544, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cd0 00010000:00010000:1.0:1789760779.873367:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.873382:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760779.873386:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.873414:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.873433:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.880124:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986146 is committed 00002000:00010000:1.0:1789760779.880229:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77164544 -> 77168640 00002000:00010000:1.0:1789760779.880235:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150736 -> 150744 00010000:00010000:1.0:1789760779.880248:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.880261:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.880271:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.880278:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.880284:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.880293:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.880326:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.880335:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77164544->77168639] (req 77164544->77168639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.880665:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7643us (7989us total) trans 4294986146 rc 0/0 00000100:00100000:1.0:1789760779.880675:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18850 00000100:00100000:1.0:1789760779.880685:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709981117606 time 709997610584 state 0x2 ref 0 00000100:00100000:2.0:1789760779.883368:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.883445:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799808 00000100:00100000:0.0:1789760779.883456:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799808 00000100:00100000:0.0:1789760779.883512:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709981117606 time 709997610584 state 0x0 ref 1 00000100:00100000:0.0:1789760779.883526:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709997871000 time 709997610584 state 0x1 ref 1 00000100:00100000:0.0:1789760779.883559:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709997871000 time 710008166373 state 0x0 ref 1 00000100:00100000:0.0:1789760779.883596:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760779.883647:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77168640, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cd8 00010000:00010000:0.0:1789760779.883657:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.883672:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760779.883677:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.883683:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.883697:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.889926:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986147 is committed 00002000:00010000:2.0:1789760779.890045:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77168640 -> 77172736 00002000:00010000:2.0:1789760779.890050:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150744 -> 150752 00010000:00010000:2.0:1789760779.890063:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.890080:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.890088:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.890096:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.890100:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.890109:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.890128:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.890135:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0b31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77168640->77172735] (req 77168640->77172735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.890232:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503799808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6647us (6867us total) trans 4294986147 rc 0/0 00000100:00100000:2.0:1789760779.890243:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18851 00000100:00100000:2.0:1789760779.890252:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709997871000 time 710008166373 state 0x2 ref 0 00000100:00100000:0.0:1789760779.893278:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.893344:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503799936 00000100:00100000:2.0:1789760779.893354:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503799936 00000100:00100000:2.0:1789760779.893433:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 709997871000 time 710008166373 state 0x0 ref 1 00000100:00100000:2.0:1789760779.893448:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710008426789 time 710008166373 state 0x1 ref 1 00000100:00100000:2.0:1789760779.893478:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710008426789 time 710018084638 state 0x0 ref 1 00000100:00100000:2.0:1789760779.893490:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.893533:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77172736, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ce0 00010000:00010000:2.0:1789760779.893542:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.893558:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760779.893562:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.893568:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.893591:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.899289:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986148 is committed 00002000:00010000:2.0:1789760779.899913:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77172736 -> 77176832 00002000:00010000:2.0:1789760779.899919:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150752 -> 150760 00010000:00010000:2.0:1789760779.899931:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.899943:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.899957:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.899964:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.899968:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.899975:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.900043:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.900053:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77172736->77176831] (req 77172736->77176831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.900194:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503799936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6701us (6920us total) trans 4294986148 rc 0/0 00000100:00100000:2.0:1789760779.900205:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18852 00000100:00100000:2.0:1789760779.900215:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710008426789 time 710018084638 state 0x2 ref 0 00000100:00100000:0.0:1789760779.902998:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760779.903192:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800064 00000100:00100000:1.0:1789760779.903202:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800064 00000100:00100000:1.0:1789760779.903650:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710008426789 time 710018084638 state 0x0 ref 1 00000100:00100000:1.0:1789760779.903666:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710018345054 time 710018084638 state 0x1 ref 1 00000100:00100000:1.0:1789760779.903698:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710018345054 time 710028304404 state 0x0 ref 1 00000100:00100000:1.0:1789760779.903713:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.903766:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77176832, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ce8 00010000:00010000:1.0:1789760779.904996:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.905023:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760779.905027:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.905036:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.905052:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.910921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986149 is committed 00002000:00010000:1.0:1789760779.911328:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77176832 -> 77180928 00002000:00010000:1.0:1789760779.911334:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150760 -> 150768 00010000:00010000:1.0:1789760779.911347:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.911361:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.911369:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.911377:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.911382:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.911409:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.911427:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.911436:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77176832->77180927] (req 77176832->77180927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.911592:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7868us (8599us total) trans 4294986149 rc 0/0 00000100:00100000:1.0:1789760779.911603:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18853 00000100:00100000:1.0:1789760779.911613:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710018345054 time 710028304404 state 0x2 ref 0 00000100:00100000:1.0:1789760779.916804:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.917009:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800192 00000100:00100000:3.0:1789760779.917019:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800192 00000100:00100000:3.0:1789760779.917074:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710018345054 time 710028304404 state 0x0 ref 1 00000100:00100000:3.0:1789760779.917100:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710028564820 time 710028304404 state 0x1 ref 1 00000100:00100000:3.0:1789760779.917134:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710028564820 time 710041740090 state 0x0 ref 1 00000100:00100000:3.0:1789760779.917146:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.917201:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77180928, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cf0 00010000:00010000:3.0:1789760779.917210:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.917226:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760779.917230:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.917237:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.917255:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.922971:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986150 is committed 00002000:00010000:3.0:1789760779.923135:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77180928 -> 77185024 00002000:00010000:3.0:1789760779.923140:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150768 -> 150776 00010000:00010000:3.0:1789760779.923153:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.923166:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.923175:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.923182:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.923186:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.923196:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.923217:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.923236:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77180928->77185023] (req 77180928->77185023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.923349:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6200us (6550us total) trans 4294986150 rc 0/0 00000100:00100000:3.0:1789760779.923358:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18854 00000100:00100000:3.0:1789760779.923366:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710028564820 time 710041740090 state 0x2 ref 0 00000100:00100000:0.0:1789760779.926118:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.926270:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800320 00000100:00100000:2.0:1789760779.926279:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800320 00000100:00100000:2.0:1789760779.926331:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710028564820 time 710041740090 state 0x0 ref 1 00000100:00100000:2.0:1789760779.926345:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710042000506 time 710041740090 state 0x1 ref 1 00000100:00100000:2.0:1789760779.926377:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710042000506 time 710050984278 state 0x0 ref 1 00000100:00100000:2.0:1789760779.926420:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.926467:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77185024, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24cf8 00010000:00010000:2.0:1789760779.926477:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.926491:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760779.926496:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.926503:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.926516:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.931251:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986151 is committed 00002000:00010000:2.0:1789760779.931355:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77185024 -> 77189120 00002000:00010000:2.0:1789760779.931361:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150776 -> 150784 00010000:00010000:2.0:1789760779.931376:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.931429:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.931440:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.931448:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.931454:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.931465:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.931485:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.931494:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77185024->77189119] (req 77185024->77189119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.931599:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5193us (5472us total) trans 4294986151 rc 0/0 00000100:00100000:2.0:1789760779.931610:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18855 00000100:00100000:2.0:1789760779.931695:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710042000506 time 710050984278 state 0x2 ref 0 00000100:00100000:0.0:1789760779.938741:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.939024:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800448 00000100:00100000:2.0:1789760779.939035:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800448 00000100:00100000:2.0:1789760779.939088:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710042000506 time 710050984278 state 0x0 ref 1 00000100:00100000:2.0:1789760779.939103:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710051244694 time 710050984278 state 0x1 ref 1 00000100:00100000:2.0:1789760779.939131:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710051244694 time 710063738301 state 0x0 ref 1 00000100:00100000:2.0:1789760779.939146:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.939189:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77189120, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d00 00010000:00010000:2.0:1789760779.939198:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.939213:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760779.939218:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.939224:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.939239:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.948217:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986152 is committed 00002000:00010000:2.0:1789760779.948775:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77189120 -> 77193216 00002000:00010000:2.0:1789760779.948780:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150784 -> 150792 00010000:00010000:2.0:1789760779.948794:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.948807:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.948814:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.948822:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.948827:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.948836:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.948856:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.948863:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77189120->77193215] (req 77189120->77193215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.949014:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9864us (10279us total) trans 4294986152 rc 0/0 00000100:00100000:2.0:1789760779.949025:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18856 00000100:00100000:2.0:1789760779.949035:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710051244694 time 710063738301 state 0x2 ref 0 00000100:00100000:1.0:1789760779.957372:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760779.957906:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800576 00000100:00100000:0.0:1789760779.957918:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800576 00000100:00100000:1.0:1789760779.958066:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710051244694 time 710063738301 state 0x0 ref 1 00000100:00100000:1.0:1789760779.958080:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710063998717 time 710063738301 state 0x1 ref 1 00000100:00100000:1.0:1789760779.958207:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710063998717 time 710082812161 state 0x0 ref 1 00000100:00100000:1.0:1789760779.958220:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760779.958273:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77193216, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d08 00010000:00010000:1.0:1789760779.958283:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.958299:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760779.958303:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.958309:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.958323:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.963207:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986153 is committed 00002000:00010000:1.0:1789760779.963444:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77193216 -> 77197312 00002000:00010000:1.0:1789760779.963450:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150792 -> 150800 00010000:00010000:1.0:1789760779.963463:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.963476:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.963485:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.963492:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760779.963497:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.963505:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.963527:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.963534:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f0b5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77193216->77197311] (req 77193216->77197311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.963637:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5414us (6268us total) trans 4294986153 rc 0/0 00000100:00100000:1.0:1789760779.963647:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18857 00000100:00100000:1.0:1789760779.963656:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710063998717 time 710082812161 state 0x2 ref 0 00000100:00100000:0.0:1789760779.969961:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760779.970078:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800704 00000100:00100000:3.0:1789760779.970088:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800704 00000100:00100000:3.0:1789760779.970144:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710063998717 time 710082812161 state 0x0 ref 1 00000100:00100000:3.0:1789760779.970158:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710083072577 time 710082812161 state 0x1 ref 1 00000100:00100000:3.0:1789760779.970222:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710083072577 time 710094828870 state 0x0 ref 1 00000100:00100000:3.0:1789760779.970235:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760779.970292:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77197312, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d10 00010000:00010000:3.0:1789760779.970302:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.970317:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760779.970321:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.970327:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.970341:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.975419:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986154 is committed 00002000:00010000:3.0:1789760779.975542:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77197312 -> 77201408 00002000:00010000:3.0:1789760779.975546:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150800 -> 150808 00010000:00010000:3.0:1789760779.975556:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.975568:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.975577:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.975660:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760779.975665:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.975675:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.975696:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.975704:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77197312->77201407] (req 77197312->77201407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.975856:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5617us (5896us total) trans 4294986154 rc 0/0 00000100:00100000:3.0:1789760779.975867:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18858 00000100:00100000:3.0:1789760779.975885:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710083072577 time 710094828870 state 0x2 ref 0 00000100:00100000:0.0:1789760779.984102:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760779.984201:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800832 00000100:00100000:2.0:1789760779.984210:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800832 00000100:00100000:2.0:1789760779.984264:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710083072577 time 710094828870 state 0x0 ref 1 00000100:00100000:2.0:1789760779.984280:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710095089286 time 710094828870 state 0x1 ref 1 00000100:00100000:2.0:1789760779.984383:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710095089286 time 710108988292 state 0x0 ref 1 00000100:00100000:2.0:1789760779.984430:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760779.984483:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77201408, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d18 00010000:00010000:2.0:1789760779.984494:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.984511:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760779.984516:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.984523:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.984538:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.990483:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986155 is committed 00002000:00010000:2.0:1789760779.992750:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77201408 -> 77205504 00002000:00010000:2.0:1789760779.992762:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150808 -> 150816 00010000:00010000:2.0:1789760779.992785:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.992804:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.992813:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.992820:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760779.992824:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.992838:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.992858:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.992865:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77201408->77205503] (req 77201408->77205503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.993000:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503800832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8569us (8904us total) trans 4294986155 rc 0/0 00000100:00100000:2.0:1789760779.993011:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18859 00000100:00100000:2.0:1789760779.993021:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710095089286 time 710108988292 state 0x2 ref 0 00000100:00100000:1.0:1789760780.005242:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.005416:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503800960 00000100:00100000:2.0:1789760780.005425:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503800960 00000100:00100000:2.0:1789760780.005478:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710095089286 time 710108988292 state 0x0 ref 1 00000100:00100000:2.0:1789760780.005490:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710109248708 time 710108988292 state 0x1 ref 1 00000100:00100000:2.0:1789760780.005522:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710109248708 time 710130128628 state 0x0 ref 1 00000100:00100000:2.0:1789760780.005533:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.005578:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77205504, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d20 00010000:00010000:2.0:1789760780.005597:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005612:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.005616:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.005631:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005644:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760780.006528:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760780.011243:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986156 is committed 00002000:00010000:2.0:1789760780.011360:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77205504 -> 77209600 00002000:00010000:2.0:1789760780.011365:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150816 -> 150824 00010000:00010000:2.0:1789760780.011378:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.011406:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.011415:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.011423:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.011428:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.011437:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.011465:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.011474:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0b70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77205504->77209599] (req 77205504->77209599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.011729:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503800960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6191us (6490us total) trans 4294986156 rc 0/0 00000100:00100000:2.0:1789760780.011740:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18860 00000100:00100000:2.0:1789760780.011750:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710109248708 time 710130128628 state 0x2 ref 0 00000100:00100000:0.0:1789760780.014997:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.015093:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801088 00000100:00100000:1.0:1789760780.015104:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801088 00000100:00100000:1.0:1789760780.015161:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710109248708 time 710130128628 state 0x0 ref 1 00000100:00100000:1.0:1789760780.015176:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710130389044 time 710130128628 state 0x1 ref 1 00000100:00100000:1.0:1789760780.015208:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710130389044 time 710139815301 state 0x0 ref 1 00000100:00100000:1.0:1789760780.015220:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503801088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.015265:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77209600, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d28 00010000:00010000:1.0:1789760780.015274:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.015289:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.015293:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.015299:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.015311:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.020458:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986157 is committed 00002000:00010000:1.0:1789760780.020750:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77209600 -> 77213696 00002000:00010000:1.0:1789760780.020756:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150824 -> 150832 00010000:00010000:1.0:1789760780.020769:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.020782:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.020791:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.020798:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.020804:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.020813:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.020833:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.020841:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f0b77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77209600->77213695] (req 77209600->77213695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.023120:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503801088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7888us (8119us total) trans 4294986157 rc 0/0 00000100:00100000:1.0:1789760780.023133:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18861 00000100:00100000:1.0:1789760780.023143:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710130389044 time 710139815301 state 0x2 ref 0 00000100:00100000:0.0:1789760780.030226:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.030381:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801216 00000100:00100000:3.0:1789760780.030466:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801216 00000100:00100000:3.0:1789760780.030718:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710130389044 time 710139815301 state 0x0 ref 1 00000100:00100000:3.0:1789760780.030735:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710140075717 time 710139815301 state 0x1 ref 1 00000100:00100000:3.0:1789760780.030772:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710140075717 time 710155378857 state 0x0 ref 1 00000100:00100000:3.0:1789760780.030786:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503801216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.030838:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77213696, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d30 00010000:00010000:3.0:1789760780.030849:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.030866:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.030871:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.030877:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.030893:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.035330:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986158 is committed 00002000:00010000:3.0:1789760780.035731:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77213696 -> 77217792 00002000:00010000:3.0:1789760780.035737:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150832 -> 150840 00010000:00010000:3.0:1789760780.035752:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.035763:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.035772:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.035779:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.035784:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.035793:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.035811:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.035818:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77213696->77217791] (req 77213696->77217791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.035908:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503801216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5121us (5687us total) trans 4294986158 rc 0/0 00000100:00100000:3.0:1789760780.035919:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18862 00000100:00100000:3.0:1789760780.035928:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710140075717 time 710155378857 state 0x2 ref 0 00000100:00100000:2.0:1789760780.045292:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760780.046288:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801344 00000100:00100000:0.0:1789760780.046300:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801344 00000100:00100000:0.0:1789760780.046463:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710140075717 time 710155378857 state 0x0 ref 1 00000100:00100000:0.0:1789760780.046478:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710155639273 time 710155378857 state 0x1 ref 1 00000100:00100000:0.0:1789760780.046640:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710155639273 time 710171109733 state 0x0 ref 1 00000100:00100000:0.0:1789760780.046655:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503801344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760780.046700:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77217792, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d38 00010000:00010000:0.0:1789760780.046710:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.046726:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760780.046730:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.046736:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.046749:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.054118:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986159 is committed 00002000:00010000:0.0:1789760780.055905:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77217792 -> 77221888 00002000:00010000:0.0:1789760780.055914:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150840 -> 150848 00010000:00010000:0.0:1789760780.056030:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.056044:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.056053:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.056061:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.056066:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.056074:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.056093:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.056100:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0b85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77217792->77221887] (req 77217792->77221887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.056260:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503801344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9603us (10974us total) trans 4294986159 rc 0/0 00000100:00100000:0.0:1789760780.056271:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18863 00000100:00100000:0.0:1789760780.056280:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710155639273 time 710171109733 state 0x2 ref 0 00000100:00100000:0.0:1789760780.067682:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.067981:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801472 00000100:00100000:2.0:1789760780.067992:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801472 00000100:00100000:2.0:1789760780.068049:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710155639273 time 710171109733 state 0x0 ref 1 00000100:00100000:2.0:1789760780.068068:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710171370149 time 710171109733 state 0x1 ref 1 00000100:00100000:2.0:1789760780.068096:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710171370149 time 710192703324 state 0x0 ref 1 00000100:00100000:2.0:1789760780.068107:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503801472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.068152:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77221888, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d40 00010000:00010000:2.0:1789760780.068161:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.068176:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.068180:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.068187:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.068201:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.075450:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986160 is committed 00002000:00010000:2.0:1789760780.077019:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77221888 -> 77225984 00002000:00010000:2.0:1789760780.077029:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150848 -> 150856 00010000:00010000:2.0:1789760780.077044:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.077057:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.077065:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.077072:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.077078:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.077086:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.077106:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.077114:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77221888->77225983] (req 77221888->77225983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.077215:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503801472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9105us (9538us total) trans 4294986160 rc 0/0 00000100:00100000:2.0:1789760780.077225:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18864 00000100:00100000:2.0:1789760780.077234:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710171370149 time 710192703324 state 0x2 ref 0 00000100:00100000:3.0:1789760780.082064:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.082255:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801600 00000100:00100000:2.0:1789760780.082264:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801600 00000100:00100000:2.0:1789760780.082319:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710171370149 time 710192703324 state 0x0 ref 1 00000100:00100000:2.0:1789760780.082333:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710192963740 time 710192703324 state 0x1 ref 1 00000100:00100000:2.0:1789760780.082362:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710192963740 time 710206969023 state 0x0 ref 1 00000100:00100000:2.0:1789760780.082374:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503801600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.082440:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77225984, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d48 00010000:00010000:2.0:1789760780.082449:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.082465:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.082470:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.082476:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.082489:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.088623:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986161 is committed 00002000:00010000:2.0:1789760780.088797:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77225984 -> 77230080 00002000:00010000:2.0:1789760780.088803:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150856 -> 150864 00010000:00010000:2.0:1789760780.088814:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.088827:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.088836:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.088844:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.088849:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.088858:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.088878:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.088891:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0b93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77225984->77230079] (req 77225984->77230079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.088980:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503801600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6603us (6920us total) trans 4294986161 rc 0/0 00000100:00100000:2.0:1789760780.088991:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18865 00000100:00100000:2.0:1789760780.089001:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710192963740 time 710206969023 state 0x2 ref 0 00000100:00100000:1.0:1789760780.094842:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.094911:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801728 00000100:00100000:3.0:1789760780.094922:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801728 00000100:00100000:3.0:1789760780.094975:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710192963740 time 710206969023 state 0x0 ref 1 00000100:00100000:3.0:1789760780.094990:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710207229439 time 710206969023 state 0x1 ref 1 00000100:00100000:3.0:1789760780.095029:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710207229439 time 710219635678 state 0x0 ref 1 00000100:00100000:3.0:1789760780.095042:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503801728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.095088:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77230080, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d50 00010000:00010000:3.0:1789760780.095098:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.095113:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.095118:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.095124:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.095138:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.103544:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986162 is committed 00002000:00010000:3.0:1789760780.103743:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77230080 -> 77234176 00002000:00010000:3.0:1789760780.103749:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150864 -> 150872 00010000:00010000:3.0:1789760780.103766:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.103783:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.103791:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.103798:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.103802:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.103811:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.103830:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.103837:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0b9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77230080->77234175] (req 77230080->77234175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.103949:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503801728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8897us (9104us total) trans 4294986162 rc 0/0 00000100:00100000:3.0:1789760780.103960:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18866 00000100:00100000:3.0:1789760780.103971:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710207229439 time 710219635678 state 0x2 ref 0 00000100:00100000:0.0:1789760780.106998:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.107108:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801856 00000100:00100000:2.0:1789760780.107118:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801856 00000100:00100000:2.0:1789760780.107182:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710207229439 time 710219635678 state 0x0 ref 1 00000100:00100000:2.0:1789760780.107198:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710219896094 time 710219635678 state 0x1 ref 1 00000100:00100000:2.0:1789760780.107229:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710219896094 time 710231835214 state 0x0 ref 1 00000100:00100000:2.0:1789760780.107242:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503801856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.107289:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77234176, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d58 00010000:00010000:2.0:1789760780.107298:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107314:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.107319:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.107325:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107345:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.117162:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986163 is committed 00002000:00010000:3.0:1789760780.117376:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77234176 -> 77238272 00002000:00010000:3.0:1789760780.117382:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150872 -> 150880 00010000:00010000:3.0:1789760780.117414:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.117426:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.117436:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.117442:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.117447:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.117456:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.117476:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.117483:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ba1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77234176->77238271] (req 77234176->77238271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.117715:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503801856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10470us (10721us total) trans 4294986163 rc 0/0 00000100:00100000:3.0:1789760780.117730:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18867 00000100:00100000:3.0:1789760780.117741:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710219896094 time 710231835214 state 0x2 ref 0 00000100:00100000:0.0:1789760780.121059:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.121171:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503801984 00000100:00100000:2.0:1789760780.121181:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503801984 00000100:00100000:2.0:1789760780.121236:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710219896094 time 710231835214 state 0x0 ref 1 00000100:00100000:2.0:1789760780.121251:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710232095630 time 710231835214 state 0x1 ref 1 00000100:00100000:2.0:1789760780.121312:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710232095630 time 710245918115 state 0x0 ref 1 00000100:00100000:2.0:1789760780.121325:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503801984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.121371:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77238272, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d60 00010000:00010000:2.0:1789760780.121379:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.121425:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.121430:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.121437:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.121452:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.127541:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986164 is committed 00002000:00010000:0.0:1789760780.128130:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77238272 -> 77242368 00002000:00010000:0.0:1789760780.128136:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150880 -> 150888 00010000:00010000:0.0:1789760780.128150:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.128169:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.128178:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.128186:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.128190:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.128199:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.128219:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.128227:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0ba8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77238272->77242367] (req 77238272->77242367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.128545:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503801984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7218us (7490us total) trans 4294986164 rc 0/0 00000100:00100000:0.0:1789760780.128557:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18868 00000100:00100000:0.0:1789760780.128566:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710232095630 time 710245918115 state 0x2 ref 0 00000100:00100000:1.0:1789760780.133036:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.133315:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503802112 00000100:00100000:2.0:1789760780.133325:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503802112 00000100:00100000:2.0:1789760780.133377:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710232095630 time 710245918115 state 0x0 ref 1 00000100:00100000:2.0:1789760780.133462:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710246178531 time 710245918115 state 0x1 ref 1 00000100:00100000:2.0:1789760780.133494:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710246178531 time 710258099998 state 0x0 ref 1 00000100:00100000:2.0:1789760780.133506:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503802112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.133551:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77242368, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d68 00010000:00010000:2.0:1789760780.133560:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.133575:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.134046:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.134056:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.134073:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.140378:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986165 is committed 00002000:00010000:2.0:1789760780.140503:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77242368 -> 77246464 00002000:00010000:2.0:1789760780.140508:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150888 -> 150896 00010000:00010000:2.0:1789760780.140521:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.140533:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.140544:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.140551:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.140556:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.140565:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.140604:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.140613:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0baf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77242368->77246463] (req 77242368->77246463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.140823:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503802112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7314us (7840us total) trans 4294986165 rc 0/0 00000100:00100000:2.0:1789760780.140834:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18869 00000100:00100000:2.0:1789760780.140845:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710246178531 time 710258099998 state 0x2 ref 0 00000100:00100000:2.0:1789760780.148261:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.148896:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503802240 00000100:00100000:3.0:1789760780.148906:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503802240 00000100:00100000:3.0:1789760780.148962:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710246178531 time 710258099998 state 0x0 ref 1 00000100:00100000:3.0:1789760780.148978:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710258360414 time 710258099998 state 0x1 ref 1 00000100:00100000:3.0:1789760780.149970:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710258360414 time 710274569431 state 0x0 ref 1 00000100:00100000:3.0:1789760780.149997:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503802240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.150058:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77246464, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d70 00010000:00010000:3.0:1789760780.150068:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.150084:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.150089:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.150096:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.150112:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.157202:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986166 is committed 00002000:00010000:3.0:1789760780.157576:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77246464 -> 77250560 00002000:00010000:3.0:1789760780.157589:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150896 -> 150904 00010000:00010000:3.0:1789760780.157602:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.157620:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.157629:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.157636:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.157641:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.157650:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.157788:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.157797:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77246464->77250559] (req 77246464->77250559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.157912:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503802240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7921us (9732us total) trans 4294986166 rc 0/0 00000100:00100000:3.0:1789760780.157920:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18870 00000100:00100000:3.0:1789760780.157928:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710258360414 time 710274569431 state 0x2 ref 0 00000100:00100000:0.0:1789760780.160522:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.160701:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503802368 00000100:00100000:3.0:1789760780.160709:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503802368 00000100:00100000:3.0:1789760780.160762:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710258360414 time 710274569431 state 0x0 ref 1 00000100:00100000:3.0:1789760780.160777:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710274829847 time 710274569431 state 0x1 ref 1 00000100:00100000:2.0:1789760780.160885:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710274829847 time 710285488804 state 0x0 ref 1 00000100:00100000:2.0:1789760780.160903:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503802368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.160947:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77250560, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d78 00010000:00010000:2.0:1789760780.160955:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.160970:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.160975:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.160981:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.160994:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.165943:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986167 is committed 00002000:00010000:2.0:1789760780.166740:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77250560 -> 77254656 00002000:00010000:2.0:1789760780.166746:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150904 -> 150912 00010000:00010000:2.0:1789760780.166761:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.166774:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.166782:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.166789:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.166794:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.166803:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.166824:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.166831:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77250560->77254655] (req 77250560->77254655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.166979:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503802368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6073us (6460us total) trans 4294986167 rc 0/0 00000100:00100000:2.0:1789760780.167001:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18871 00000100:00100000:2.0:1789760780.167011:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710274829847 time 710285488804 state 0x2 ref 0 00000100:00100000:1.0:1789760780.171133:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760780.171212:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503802496 00000100:00100000:0.0:1789760780.171223:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503802496 00000100:00100000:0.0:1789760780.171274:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710274829847 time 710285488804 state 0x0 ref 1 00000100:00100000:0.0:1789760780.171287:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710285749220 time 710285488804 state 0x1 ref 1 00000100:00100000:0.0:1789760780.171314:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710285749220 time 710295920518 state 0x0 ref 1 00000100:00100000:0.0:1789760780.171325:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503802496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760780.171372:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77254656, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d80 00010000:00010000:0.0:1789760780.171381:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.171431:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760780.171436:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.171442:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.171455:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.178327:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986168 is committed 00002000:00010000:0.0:1789760780.178783:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77254656 -> 77258752 00002000:00010000:0.0:1789760780.178790:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150912 -> 150920 00010000:00010000:0.0:1789760780.178803:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.178827:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.178838:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.178845:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.178850:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.178858:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.178877:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.178884:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0bc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77254656->77258751] (req 77254656->77258751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.178976:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503802496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7647us (7844us total) trans 4294986168 rc 0/0 00000100:00100000:0.0:1789760780.178986:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18872 00000100:00100000:0.0:1789760780.178995:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710285749220 time 710295920518 state 0x2 ref 0 00000100:00100000:0.0:1789760780.182439:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.182515:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503802624 00000100:00100000:3.0:1789760780.182525:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503802624 00000100:00100000:2.0:1789760780.182930:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710285749220 time 710295920518 state 0x0 ref 1 00000100:00100000:2.0:1789760780.182945:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710296180934 time 710295920518 state 0x1 ref 1 00000100:00100000:2.0:1789760780.183034:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710296180934 time 710307639552 state 0x0 ref 1 00000100:00100000:2.0:1789760780.183047:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503802624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.183098:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77258752, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d88 00010000:00010000:2.0:1789760780.183107:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.183124:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.183128:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.183134:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.183148:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.189443:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986169 is committed 00002000:00010000:2.0:1789760780.189922:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77258752 -> 77262848 00002000:00010000:2.0:1789760780.189929:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150920 -> 150928 00010000:00010000:2.0:1789760780.189941:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.189951:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.189956:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.189963:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.189967:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.190022:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.190051:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.190057:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bcb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77258752->77262847] (req 77258752->77262847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.190144:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503802624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7094us (7710us total) trans 4294986169 rc 0/0 00000100:00100000:2.0:1789760780.190154:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18873 00000100:00100000:2.0:1789760780.190163:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710296180934 time 710307639552 state 0x2 ref 0 00000100:00100000:3.0:1789760780.196308:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.196541:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503802752 00000100:00100000:1.0:1789760780.196551:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503802752 00000100:00100000:1.0:1789760780.196626:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710296180934 time 710307639552 state 0x0 ref 1 00000100:00100000:1.0:1789760780.196643:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710307899968 time 710307639552 state 0x1 ref 1 00000100:00100000:1.0:1789760780.196668:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710307899968 time 710321274651 state 0x0 ref 1 00000100:00100000:1.0:1789760780.196679:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503802752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.196733:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77262848, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d90 00010000:00010000:1.0:1789760780.196743:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.196759:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.196763:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.196770:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.196785:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.210211:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986170 is committed 00002000:00010000:1.0:1789760780.210317:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77262848 -> 77266944 00002000:00010000:1.0:1789760780.210323:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150928 -> 150936 00010000:00010000:1.0:1789760780.210337:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.210351:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.210360:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.210382:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.210405:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.210414:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.210435:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.210443:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f0bd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77262848->77266943] (req 77262848->77266943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.210539:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503802752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13857us (14238us total) trans 4294986170 rc 0/0 00000100:00100000:1.0:1789760780.210549:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18874 00000100:00100000:1.0:1789760780.210559:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710307899968 time 710321274651 state 0x2 ref 0 00000100:00100000:1.0:1789760780.217483:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.217801:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503802880 00000100:00100000:2.0:1789760780.217812:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503802880 00000100:00100000:2.0:1789760780.217869:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710307899968 time 710321274651 state 0x0 ref 1 00000100:00100000:2.0:1789760780.217963:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710321535067 time 710321274651 state 0x1 ref 1 00000100:00100000:2.0:1789760780.218042:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710321535067 time 710342648425 state 0x0 ref 1 00000100:00100000:2.0:1789760780.218057:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503802880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.218970:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77266944, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24d98 00010000:00010000:2.0:1789760780.218985:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.219006:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.219010:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.219016:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.219029:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.225447:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986171 is committed 00002000:00010000:2.0:1789760780.225573:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77266944 -> 77271040 00002000:00010000:2.0:1789760780.225586:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150936 -> 150944 00010000:00010000:2.0:1789760780.225601:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.225613:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.225622:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.225630:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.225634:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.225643:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.225669:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.225677:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0bd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77266944->77271039] (req 77266944->77271039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.225773:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503802880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7715us (8295us total) trans 4294986171 rc 0/0 00000100:00100000:2.0:1789760780.225784:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18875 00000100:00100000:2.0:1789760780.225795:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710321535067 time 710342648425 state 0x2 ref 0 00000100:00100000:0.0:1789760780.235466:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.235622:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803008 00000100:00100000:2.0:1789760780.235631:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803008 00000100:00100000:2.0:1789760780.235686:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710321535067 time 710342648425 state 0x0 ref 1 00000100:00100000:2.0:1789760780.235702:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710342908841 time 710342648425 state 0x1 ref 1 00000100:00100000:2.0:1789760780.235722:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710342908841 time 710360329528 state 0x0 ref 1 00000100:00100000:2.0:1789760780.235733:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc16c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.235779:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77271040, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24da0 00010000:00010000:2.0:1789760780.235789:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.235806:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.235812:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.235818:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.235831:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.241974:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986172 is committed 00002000:00010000:0.0:1789760780.243150:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77271040 -> 77275136 00002000:00010000:0.0:1789760780.243158:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150944 -> 150952 00010000:00010000:0.0:1789760780.243171:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243184:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243192:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243200:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.243204:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243213:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243231:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243239:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0be0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77271040->77275135] (req 77271040->77275135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.243495:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc16c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7758us (8035us total) trans 4294986172 rc 0/0 00000100:00100000:0.0:1789760780.243506:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18876 00000100:00100000:0.0:1789760780.243516:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710342908841 time 710360329528 state 0x2 ref 0 00000100:00100000:3.0:1789760780.249140:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760780.251709:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803136 00000100:00100000:0.0:1789760780.251723:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803136 00000100:00100000:0.0:1789760780.251788:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710342908841 time 710360329528 state 0x0 ref 1 00000100:00100000:0.0:1789760780.251804:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710360589944 time 710360329528 state 0x1 ref 1 00000100:00100000:0.0:1789760780.251906:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710360589944 time 710376512344 state 0x0 ref 1 00000100:00100000:0.0:1789760780.251921:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503803136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760780.251969:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77275136, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24da8 00010000:00010000:0.0:1789760780.251979:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.251993:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760780.251998:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.252005:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.252018:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.261645:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986173 is committed 00002000:00010000:0.0:1789760780.261805:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77275136 -> 77279232 00002000:00010000:0.0:1789760780.261810:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150952 -> 150960 00010000:00010000:0.0:1789760780.261822:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.261840:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.261849:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.261855:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.261860:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.261869:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.261886:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.261894:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0be7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77275136->77279231] (req 77275136->77279231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.261982:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503803136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10060us (12848us total) trans 4294986173 rc 0/0 00000100:00100000:0.0:1789760780.261990:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18877 00000100:00100000:0.0:1789760780.261999:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710360589944 time 710376512344 state 0x2 ref 0 00000100:00100000:1.0:1789760780.267169:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.267339:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803264 00000100:00100000:3.0:1789760780.267349:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803264 00000100:00100000:3.0:1789760780.267435:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710360589944 time 710376512344 state 0x0 ref 1 00000100:00100000:3.0:1789760780.267450:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710376772760 time 710376512344 state 0x1 ref 1 00000100:00100000:3.0:1789760780.267482:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710376772760 time 710392088898 state 0x0 ref 1 00000100:00100000:3.0:1789760780.267495:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.267539:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77279232, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24db0 00010000:00010000:3.0:1789760780.267547:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.267563:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.267567:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.267573:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.267830:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.281441:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986174 is committed 00002000:00010000:1.0:1789760780.281710:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77279232 -> 77283328 00002000:00010000:1.0:1789760780.281717:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150960 -> 150968 00010000:00010000:1.0:1789760780.281730:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.281743:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.281751:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.281759:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.281764:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.281773:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.281797:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.281804:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0bee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77279232->77283327] (req 77279232->77283327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.281958:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14460us (14794us total) trans 4294986174 rc 0/0 00000100:00100000:1.0:1789760780.281969:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18878 00000100:00100000:1.0:1789760780.281978:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710376772760 time 710392088898 state 0x2 ref 0 00000100:00100000:3.0:1789760780.285707:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.285816:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803392 00000100:00100000:2.0:1789760780.285826:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803392 00000100:00100000:2.0:1789760780.285896:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710376772760 time 710392088898 state 0x0 ref 1 00000100:00100000:2.0:1789760780.285910:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710392349314 time 710392088898 state 0x1 ref 1 00000100:00100000:2.0:1789760780.285979:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710392349314 time 710410585076 state 0x0 ref 1 00000100:00100000:2.0:1789760780.285992:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503803392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.286046:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77283328, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24db8 00010000:00010000:2.0:1789760780.286054:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.286069:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.286073:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.286079:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.286101:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.292085:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986175 is committed 00002000:00010000:2.0:1789760780.292287:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77283328 -> 77287424 00002000:00010000:2.0:1789760780.292292:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150968 -> 150976 00010000:00010000:2.0:1789760780.292304:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.292316:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.292325:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.292332:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.292337:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.292346:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.292364:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.292372:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0bf5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77283328->77287423] (req 77283328->77287423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.292545:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503803392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6550us (6842us total) trans 4294986175 rc 0/0 00000100:00100000:2.0:1789760780.292557:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18879 00000100:00100000:2.0:1789760780.292566:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710392349314 time 710410585076 state 0x2 ref 0 00000100:00100000:3.0:1789760780.296251:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760780.298109:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803520 00000100:00100000:0.0:1789760780.298124:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803520 00000100:00100000:1.0:1789760780.298265:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710392349314 time 710410585076 state 0x0 ref 1 00000100:00100000:1.0:1789760780.298280:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710410845492 time 710410585076 state 0x1 ref 1 00000100:00100000:1.0:1789760780.298310:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710410845492 time 710422916989 state 0x0 ref 1 00000100:00100000:1.0:1789760780.298324:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.298373:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77287424, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24dc0 00010000:00010000:1.0:1789760780.298382:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.298416:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.298421:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.298429:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.298442:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.304888:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986176 is committed 00002000:00010000:1.0:1789760780.304979:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77287424 -> 77291520 00002000:00010000:1.0:1789760780.304985:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150976 -> 150984 00010000:00010000:1.0:1789760780.304995:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305004:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305010:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305014:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.305018:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305024:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305038:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.305046:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0bfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77287424->77291519] (req 77287424->77291519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.305125:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6801us (8881us total) trans 4294986176 rc 0/0 00000100:00100000:1.0:1789760780.305133:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18880 00000100:00100000:1.0:1789760780.305141:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710410845492 time 710422916989 state 0x2 ref 0 00000100:00100000:1.0:1789760780.312788:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760780.312866:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803648 00000100:00100000:0.0:1789760780.312874:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803648 00000100:00100000:0.0:1789760780.312961:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710410845492 time 710422916989 state 0x0 ref 1 00000100:00100000:0.0:1789760780.312976:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710423177405 time 710422916989 state 0x1 ref 1 00000100:00100000:0.0:1789760780.313007:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710423177405 time 710437613932 state 0x0 ref 1 00000100:00100000:0.0:1789760780.313019:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760780.313077:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77291520, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24dc8 00010000:00010000:0.0:1789760780.313086:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.313283:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760780.313287:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.313293:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.313306:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.322270:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986177 is committed 00002000:00010000:1.0:1789760780.324554:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77291520 -> 77295616 00002000:00010000:1.0:1789760780.324564:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150984 -> 150992 00010000:00010000:1.0:1789760780.324586:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.324601:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.324610:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.324617:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.324622:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.325075:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.325098:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.325106:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0c03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77291520->77295615] (req 77291520->77295615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.325210:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503803648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12183us (12418us total) trans 4294986177 rc 0/0 00000100:00100000:1.0:1789760780.325221:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18881 00000100:00100000:1.0:1789760780.325231:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710423177405 time 710437613932 state 0x2 ref 0 00000100:00100000:3.0:1789760780.329230:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.329337:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803776 00000100:00100000:1.0:1789760780.329347:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803776 00000100:00100000:1.0:1789760780.329425:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710423177405 time 710437613932 state 0x0 ref 1 00000100:00100000:1.0:1789760780.329441:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710437874348 time 710437613932 state 0x1 ref 1 00000100:00100000:1.0:1789760780.329462:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710437874348 time 710454068404 state 0x0 ref 1 00000100:00100000:1.0:1789760780.329475:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503803776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.329524:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77295616, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24dd0 00010000:00010000:1.0:1789760780.329533:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.329550:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.329556:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.329561:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.329575:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.335068:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986178 is committed 00002000:00010000:1.0:1789760780.335815:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77295616 -> 77299712 00002000:00010000:1.0:1789760780.335821:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 150992 -> 151000 00010000:00010000:1.0:1789760780.335833:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.335845:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.335855:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.335863:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.335868:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.335876:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.335894:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.335907:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77295616->77299711] (req 77295616->77299711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.336137:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503803776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6658us (6910us total) trans 4294986178 rc 0/0 00000100:00100000:1.0:1789760780.336148:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18882 00000100:00100000:1.0:1789760780.336158:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710437874348 time 710454068404 state 0x2 ref 0 00000100:00100000:3.0:1789760780.339034:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.339112:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503803904 00000100:00100000:2.0:1789760780.339124:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503803904 00000100:00100000:2.0:1789760780.339174:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710437874348 time 710454068404 state 0x0 ref 1 00000100:00100000:2.0:1789760780.339189:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710454328820 time 710454068404 state 0x1 ref 1 00000100:00100000:2.0:1789760780.339222:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710454328820 time 710463828026 state 0x0 ref 1 00000100:00100000:2.0:1789760780.339235:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503803904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.339280:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77299712, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24dd8 00010000:00010000:2.0:1789760780.339289:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.339467:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.339473:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.339480:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.339494:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.344713:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986179 is committed 00002000:00010000:2.0:1789760780.344871:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77299712 -> 77303808 00002000:00010000:2.0:1789760780.344933:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151000 -> 151008 00010000:00010000:2.0:1789760780.344947:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.344959:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.344967:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.344975:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.344980:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.344988:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.345008:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.345016:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77299712->77303807] (req 77299712->77303807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.345104:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503803904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5868us (6084us total) trans 4294986179 rc 0/0 00000100:00100000:2.0:1789760780.345115:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18883 00000100:00100000:2.0:1789760780.345126:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710454328820 time 710463828026 state 0x2 ref 0 00000100:00100000:1.0:1789760780.349651:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.349754:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804032 00000100:00100000:3.0:1789760780.349765:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804032 00000100:00100000:3.0:1789760780.349821:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710454328820 time 710463828026 state 0x0 ref 1 00000100:00100000:3.0:1789760780.349838:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710464088442 time 710463828026 state 0x1 ref 1 00000100:00100000:3.0:1789760780.349869:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710464088442 time 710474475105 state 0x0 ref 1 00000100:00100000:3.0:1789760780.349883:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.349953:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77303808, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24de0 00010000:00010000:3.0:1789760780.349965:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.349985:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.349990:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.349996:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.350023:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.357576:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986180 is committed 00002000:00010000:3.0:1789760780.357712:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77303808 -> 77307904 00002000:00010000:3.0:1789760780.357718:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151008 -> 151016 00010000:00010000:3.0:1789760780.357731:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.357744:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.357753:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.357761:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.357766:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.357776:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.357796:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.357810:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77303808->77307903] (req 77303808->77307903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.357919:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8035us (8274us total) trans 4294986180 rc 0/0 00000100:00100000:3.0:1789760780.357929:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18884 00000100:00100000:3.0:1789760780.357939:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710464088442 time 710474475105 state 0x2 ref 0 00000100:00100000:1.0:1789760780.361590:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.361844:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804160 00000100:00100000:2.0:1789760780.361858:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804160 00000100:00100000:2.0:1789760780.361914:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710464088442 time 710474475105 state 0x0 ref 1 00000100:00100000:2.0:1789760780.361930:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710474735521 time 710474475105 state 0x1 ref 1 00000100:00100000:2.0:1789760780.361995:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710474735521 time 710486601004 state 0x0 ref 1 00000100:00100000:2.0:1789760780.362008:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.362054:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77307904, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24de8 00010000:00010000:2.0:1789760780.362063:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.362079:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.362083:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.362090:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.362104:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.367364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986181 is committed 00002000:00010000:2.0:1789760780.367515:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77307904 -> 77312000 00002000:00010000:2.0:1789760780.367520:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151016 -> 151024 00010000:00010000:2.0:1789760780.367532:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.367545:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.367554:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.367562:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.367567:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.367576:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.367606:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.367614:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77307904->77311999] (req 77307904->77311999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.367777:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5767us (6201us total) trans 4294986181 rc 0/0 00000100:00100000:2.0:1789760780.367787:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18885 00000100:00100000:2.0:1789760780.367796:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710474735521 time 710486601004 state 0x2 ref 0 00000100:00100000:0.0:1789760780.371223:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.371477:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804288 00000100:00100000:1.0:1789760780.371487:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804288 00000100:00100000:1.0:1789760780.371744:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710474735521 time 710486601004 state 0x0 ref 1 00000100:00100000:1.0:1789760780.371764:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710486861420 time 710486601004 state 0x1 ref 1 00000100:00100000:1.0:1789760780.371835:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710486861420 time 710496441592 state 0x0 ref 1 00000100:00100000:1.0:1789760780.371861:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.371909:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77312000, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24df0 00010000:00010000:1.0:1789760780.371919:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.371939:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.371944:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.371950:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.371964:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.377477:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986182 is committed 00002000:00010000:1.0:1789760780.377609:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77312000 -> 77316096 00002000:00010000:1.0:1789760780.377614:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151024 -> 151032 00010000:00010000:1.0:1789760780.377627:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.377640:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.377649:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.377656:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.377660:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.377669:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.377687:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.377702:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77312000->77316095] (req 77312000->77316095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.377796:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5945us (6579us total) trans 4294986182 rc 0/0 00000100:00100000:1.0:1789760780.377806:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18886 00000100:00100000:1.0:1789760780.377817:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710486861420 time 710496441592 state 0x2 ref 0 00000100:00100000:0.0:1789760780.380165:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.380249:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804416 00000100:00100000:1.0:1789760780.380260:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804416 00000100:00100000:1.0:1789760780.380313:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710486861420 time 710496441592 state 0x0 ref 1 00000100:00100000:1.0:1789760780.380328:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710496702008 time 710496441592 state 0x1 ref 1 00000100:00100000:1.0:1789760780.380361:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710496702008 time 710504967713 state 0x0 ref 1 00000100:00100000:1.0:1789760780.380373:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.380448:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77316096, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24df8 00010000:00010000:1.0:1789760780.380457:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380472:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.380476:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.380481:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380497:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.388756:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986183 is committed 00002000:00010000:1.0:1789760780.389060:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77316096 -> 77320192 00002000:00010000:1.0:1789760780.389066:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151032 -> 151040 00010000:00010000:1.0:1789760780.389080:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.389093:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.389103:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.389111:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.389115:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.389124:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.389143:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.389151:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77316096->77320191] (req 77316096->77320191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.389250:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8874us (9090us total) trans 4294986183 rc 0/0 00000100:00100000:1.0:1789760780.389260:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18887 00000100:00100000:1.0:1789760780.389270:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710496702008 time 710504967713 state 0x2 ref 0 00000100:00100000:2.0:1789760780.392084:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.392474:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804544 00000100:00100000:3.0:1789760780.392485:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804544 00000100:00100000:3.0:1789760780.392542:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710496702008 time 710504967713 state 0x0 ref 1 00000100:00100000:3.0:1789760780.392557:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710505228129 time 710504967713 state 0x1 ref 1 00000100:00100000:3.0:1789760780.392591:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710505228129 time 710517197407 state 0x0 ref 1 00000100:00100000:3.0:1789760780.392604:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.393165:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77320192, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e00 00010000:00010000:3.0:1789760780.393173:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.393186:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.393190:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.393194:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.393205:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.400488:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986184 is committed 00002000:00010000:3.0:1789760780.400661:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77320192 -> 77324288 00002000:00010000:3.0:1789760780.400667:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151040 -> 151048 00010000:00010000:3.0:1789760780.400680:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.400693:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.400702:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.400715:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.400720:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.400730:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.400751:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.400764:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77320192->77324287] (req 77320192->77324287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.400868:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8262us (8789us total) trans 4294986184 rc 0/0 00000100:00100000:3.0:1789760780.400879:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18888 00000100:00100000:3.0:1789760780.400888:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710505228129 time 710517197407 state 0x2 ref 0 00000100:00100000:0.0:1789760780.405166:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.405290:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804672 00000100:00100000:2.0:1789760780.405300:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804672 00000100:00100000:2.0:1789760780.405352:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710505228129 time 710517197407 state 0x0 ref 1 00000100:00100000:2.0:1789760780.405366:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710517457823 time 710517197407 state 0x1 ref 1 00000100:00100000:2.0:1789760780.405477:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710517457823 time 710530083001 state 0x0 ref 1 00000100:00100000:2.0:1789760780.405489:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.405540:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77324288, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e08 00010000:00010000:2.0:1789760780.405549:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.405564:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.405569:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.405575:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.405605:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.413618:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986185 is committed 00002000:00010000:1.0:1789760780.413793:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77324288 -> 77328384 00002000:00010000:1.0:1789760780.413799:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151048 -> 151056 00010000:00010000:1.0:1789760780.413812:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.413825:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.413834:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.413841:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.413846:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.414015:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.414034:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.414045:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0c3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77324288->77328383] (req 77324288->77328383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.414138:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8646us (8977us total) trans 4294986185 rc 0/0 00000100:00100000:1.0:1789760780.414147:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18889 00000100:00100000:1.0:1789760780.414158:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710517457823 time 710530083001 state 0x2 ref 0 00000100:00100000:0.0:1789760780.418019:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.418067:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804800 00000100:00100000:1.0:1789760780.418078:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804800 00000100:00100000:1.0:1789760780.418139:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710517457823 time 710530083001 state 0x0 ref 1 00000100:00100000:1.0:1789760780.418155:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710530343417 time 710530083001 state 0x1 ref 1 00000100:00100000:1.0:1789760780.418184:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710530343417 time 710542790889 state 0x0 ref 1 00000100:00100000:1.0:1789760780.418197:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.418257:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77328384, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e10 00010000:00010000:1.0:1789760780.418536:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.418554:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.418559:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.418565:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.418595:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.424532:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986186 is committed 00002000:00010000:1.0:1789760780.424860:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77328384 -> 77332480 00002000:00010000:1.0:1789760780.424865:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151056 -> 151064 00010000:00010000:1.0:1789760780.424937:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.424949:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.424959:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.424965:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.424970:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.424978:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.424998:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.425006:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77328384->77332479] (req 77328384->77332479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.425136:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503804800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6936us (7120us total) trans 4294986186 rc 0/0 00000100:00100000:1.0:1789760780.425146:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18890 00000100:00100000:1.0:1789760780.425154:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710530343417 time 710542790889 state 0x2 ref 0 00000100:00100000:2.0:1789760780.429614:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.430491:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503804928 00000100:00100000:1.0:1789760780.430501:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503804928 00000100:00100000:1.0:1789760780.430559:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710530343417 time 710542790889 state 0x0 ref 1 00000100:00100000:1.0:1789760780.430586:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710543051305 time 710542790889 state 0x1 ref 1 00000100:00100000:1.0:1789760780.430774:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710543051305 time 710555379008 state 0x0 ref 1 00000100:00100000:1.0:1789760780.430787:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.430836:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77332480, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e18 00010000:00010000:1.0:1789760780.430846:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.430861:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.430865:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.430871:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.430896:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.437865:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986187 is committed 00002000:00010000:1.0:1789760780.438154:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77332480 -> 77336576 00002000:00010000:1.0:1789760780.438160:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151064 -> 151072 00010000:00010000:1.0:1789760780.438173:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.438186:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.438196:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.438203:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.438207:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.438217:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.438708:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.438719:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77332480->77336575] (req 77332480->77336575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.438854:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503804928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8064us (9246us total) trans 4294986187 rc 0/0 00000100:00100000:1.0:1789760780.438866:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18891 00000100:00100000:1.0:1789760780.438880:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710543051305 time 710555379008 state 0x2 ref 0 00000100:00100000:0.0:1789760780.442537:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.442724:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805056 00000100:00100000:3.0:1789760780.442734:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805056 00000100:00100000:3.0:1789760780.442785:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710543051305 time 710555379008 state 0x0 ref 1 00000100:00100000:3.0:1789760780.442800:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710555639424 time 710555379008 state 0x1 ref 1 00000100:00100000:3.0:1789760780.442832:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710555639424 time 710567438971 state 0x0 ref 1 00000100:00100000:3.0:1789760780.442845:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503805056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.442896:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77336576, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e20 00010000:00010000:3.0:1789760780.442905:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.442921:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.442937:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.443164:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.443182:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.450056:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986188 is committed 00002000:00010000:2.0:1789760780.450183:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77336576 -> 77340672 00002000:00010000:2.0:1789760780.450188:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151072 -> 151080 00010000:00010000:2.0:1789760780.450200:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.450212:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.450219:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.450225:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.450229:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.450238:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.450258:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.450266:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0c50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77336576->77340671] (req 77336576->77340671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.450469:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503805056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7622us (7937us total) trans 4294986188 rc 0/0 00000100:00100000:2.0:1789760780.450479:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18892 00000100:00100000:2.0:1789760780.450489:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710555639424 time 710567438971 state 0x2 ref 0 00000100:00100000:0.0:1789760780.454779:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.455021:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805184 00000100:00100000:2.0:1789760780.455031:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805184 00000100:00100000:2.0:1789760780.455082:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710555639424 time 710567438971 state 0x0 ref 1 00000100:00100000:2.0:1789760780.455097:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710567699387 time 710567438971 state 0x1 ref 1 00000100:00100000:2.0:1789760780.455173:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710567699387 time 710579778394 state 0x0 ref 1 00000100:00100000:2.0:1789760780.455186:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503805184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.455232:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77340672, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e28 00010000:00010000:2.0:1789760780.455241:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.455257:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.455262:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.455268:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.455281:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.463817:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986189 is committed 00002000:00010000:1.0:1789760780.463969:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77340672 -> 77344768 00002000:00010000:1.0:1789760780.463975:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151080 -> 151088 00010000:00010000:1.0:1789760780.463988:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464011:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464020:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464027:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.464031:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464038:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464053:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464059:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0c57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77340672->77344767] (req 77340672->77344767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.464142:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503805184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8954us (9367us total) trans 4294986189 rc 0/0 00000100:00100000:1.0:1789760780.464152:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18893 00000100:00100000:1.0:1789760780.464163:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710567699387 time 710579778394 state 0x2 ref 0 00000100:00100000:0.0:1789760780.468606:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.468668:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805312 00000100:00100000:1.0:1789760780.468678:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805312 00000100:00100000:1.0:1789760780.468732:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710567699387 time 710579778394 state 0x0 ref 1 00000100:00100000:1.0:1789760780.468747:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710580038810 time 710579778394 state 0x1 ref 1 00000100:00100000:1.0:1789760780.468778:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710580038810 time 710593384777 state 0x0 ref 1 00000100:00100000:1.0:1789760780.468790:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503805312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.468836:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77344768, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e30 00010000:00010000:1.0:1789760780.468845:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468860:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.468864:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.468870:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468884:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.477637:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986190 is committed 00002000:00010000:1.0:1789760780.477978:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77344768 -> 77348864 00002000:00010000:1.0:1789760780.477983:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151088 -> 151096 00010000:00010000:1.0:1789760780.477996:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.478009:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.478018:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.478025:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.478030:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.478039:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.478058:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.478066:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77344768->77348863] (req 77344768->77348863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.478157:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503805312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9365us (9557us total) trans 4294986190 rc 0/0 00000100:00100000:1.0:1789760780.478168:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18894 00000100:00100000:1.0:1789760780.478178:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710580038810 time 710593384777 state 0x2 ref 0 00000100:00100000:0.0:1789760780.483382:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.483748:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805440 00000100:00100000:1.0:1789760780.483758:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805440 00000100:00100000:1.0:1789760780.483810:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710580038810 time 710593384777 state 0x0 ref 1 00000100:00100000:1.0:1789760780.483824:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710593645193 time 710593384777 state 0x1 ref 1 00000100:00100000:1.0:1789760780.483902:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710593645193 time 710608507683 state 0x0 ref 1 00000100:00100000:1.0:1789760780.483914:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503805440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.483959:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77348864, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e38 00010000:00010000:1.0:1789760780.483968:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.483984:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.483988:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.483994:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.484008:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.490837:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986191 is committed 00002000:00010000:1.0:1789760780.491290:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77348864 -> 77352960 00002000:00010000:1.0:1789760780.491296:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151096 -> 151104 00010000:00010000:1.0:1789760780.491363:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.491377:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.491386:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.491482:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.491487:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.491497:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.491517:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.491525:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77348864->77352959] (req 77348864->77352959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.491653:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503805440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7736us (8274us total) trans 4294986191 rc 0/0 00000100:00100000:1.0:1789760780.491663:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18895 00000100:00100000:1.0:1789760780.491672:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710593645193 time 710608507683 state 0x2 ref 0 00000100:00100000:0.0:1789760780.494542:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.494961:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805568 00000100:00100000:2.0:1789760780.494972:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805568 00000100:00100000:2.0:1789760780.495028:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710593645193 time 710608507683 state 0x0 ref 1 00000100:00100000:2.0:1789760780.495042:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710608768099 time 710608507683 state 0x1 ref 1 00000100:00100000:2.0:1789760780.495112:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710608768099 time 710619717492 state 0x0 ref 1 00000100:00100000:2.0:1789760780.495125:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503805568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.495168:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77352960, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e40 00010000:00010000:2.0:1789760780.495177:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.495193:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.495197:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.495203:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.495216:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.498554:0:3637:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760780.498588:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ddfa03b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523610240:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760780.498783:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760780.499096:0:8432:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523610240 00000100:00100000:1.0:1789760780.499105:0:8432:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523610240 00000100:00100000:1.0:1789760780.499165:0:8432:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789760780.499189:0:8432:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523610240:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760780.499214:0:8432:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3846668288 avail 3611619328 00000100:00100000:1.0:1789760780.499274:0:8432:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523610240:12345-0@lo:13:osp-pre-0-1.0 Request processed in 100us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760780.499281:0:8432:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00080000:2.0:1789760780.499306:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760780.499333:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ddfa03b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523610240:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760780.501715:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523610368:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760780.501803:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760780.502864:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523610368 00000100:00100000:1.0:1789760780.502877:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523610368 00000100:00100000:1.0:1789760780.502949:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:1.0:1789760780.502958:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523610368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760780.502981:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3846668288 avail 3611619328 00000100:00100000:1.0:1789760780.503460:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523610368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 502us (1656us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760780.503469:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:3.0:1789760780.503566:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523610368:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1789760780.503640:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760780.505323:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986192 is committed 00002000:00010000:2.0:1789760780.505487:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77352960 -> 77357056 00002000:00010000:2.0:1789760780.505493:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151104 -> 151112 00010000:00010000:2.0:1789760780.505504:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.505517:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.505526:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.505599:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.505604:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.505614:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.505635:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.505642:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77352960->77357055] (req 77352960->77357055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.505840:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503805568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10713us (11303us total) trans 4294986192 rc 0/0 00000100:00100000:2.0:1789760780.505850:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18896 00000100:00100000:2.0:1789760780.505860:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710608768099 time 710619717492 state 0x2 ref 0 00000100:00100000:0.0:1789760780.509703:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.509774:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805696 00000100:00100000:2.0:1789760780.509783:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805696 00000100:00100000:2.0:1789760780.509833:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710608768099 time 710619717492 state 0x0 ref 1 00000100:00100000:2.0:1789760780.509848:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710619977908 time 710619717492 state 0x1 ref 1 00000100:00100000:2.0:1789760780.509874:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710619977908 time 710634481231 state 0x0 ref 1 00000100:00100000:2.0:1789760780.509903:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503805696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.509955:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77357056, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e48 00010000:00010000:2.0:1789760780.509964:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.509980:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.509985:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.509992:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.510004:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.517047:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986193 is committed 00002000:00010000:2.0:1789760780.517227:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77357056 -> 77361152 00002000:00010000:2.0:1789760780.517232:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151112 -> 151120 00010000:00010000:2.0:1789760780.517246:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.517257:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.517270:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.517277:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.517282:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.517291:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.517320:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.517328:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77357056->77361151] (req 77357056->77361151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.517493:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503805696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7593us (7794us total) trans 4294986193 rc 0/0 00000100:00100000:2.0:1789760780.517503:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18897 00000100:00100000:2.0:1789760780.517513:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710619977908 time 710634481231 state 0x2 ref 0 00000100:00100000:0.0:1789760780.524042:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.524131:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805824 00000100:00100000:1.0:1789760780.524142:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805824 00000100:00100000:1.0:1789760780.524198:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710619977908 time 710634481231 state 0x0 ref 1 00000100:00100000:1.0:1789760780.524212:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710634741647 time 710634481231 state 0x1 ref 1 00000100:00100000:1.0:1789760780.524277:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710634741647 time 710648882958 state 0x0 ref 1 00000100:00100000:1.0:1789760780.524291:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503805824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.524405:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77361152, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e50 00010000:00010000:1.0:1789760780.524415:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.524431:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.524436:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.524442:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.524455:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.530230:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986194 is committed 00002000:00010000:3.0:1789760780.530587:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77361152 -> 77365248 00002000:00010000:3.0:1789760780.530592:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151120 -> 151128 00010000:00010000:3.0:1789760780.530605:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.530746:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.530757:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.530765:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.530770:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.530785:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.530806:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.530815:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f0c7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77361152->77365247] (req 77361152->77365247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.531078:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503805824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6784us (7040us total) trans 4294986194 rc 0/0 00000100:00100000:3.0:1789760780.531089:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18898 00000100:00100000:3.0:1789760780.531099:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710634741647 time 710648882958 state 0x2 ref 0 00000100:00100000:0.0:1789760780.538332:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.538668:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503805952 00000100:00100000:2.0:1789760780.538678:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503805952 00000100:00100000:2.0:1789760780.538799:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710634741647 time 710648882958 state 0x0 ref 1 00000100:00100000:2.0:1789760780.538814:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710649143374 time 710648882958 state 0x1 ref 1 00000100:00100000:2.0:1789760780.538879:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710649143374 time 710663484501 state 0x0 ref 1 00000100:00100000:2.0:1789760780.538891:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503805952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.538945:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77365248, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e58 00010000:00010000:2.0:1789760780.538955:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.538971:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.538975:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.538981:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.538993:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.547156:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986195 is committed 00002000:00010000:1.0:1789760780.547462:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77365248 -> 77369344 00002000:00010000:1.0:1789760780.547468:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151128 -> 151136 00010000:00010000:1.0:1789760780.547482:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.547495:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.547502:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.547509:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.547514:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.547523:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.547606:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.547614:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0c81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77365248->77369343] (req 77365248->77369343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.548218:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503805952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9323us (9889us total) trans 4294986195 rc 0/0 00000100:00100000:1.0:1789760780.548230:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18899 00000100:00100000:1.0:1789760780.548240:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710649143374 time 710663484501 state 0x2 ref 0 00000100:00100000:0.0:1789760780.553779:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.553860:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806080 00000100:00100000:2.0:1789760780.553870:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806080 00000100:00100000:2.0:1789760780.553940:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710649143374 time 710663484501 state 0x0 ref 1 00000100:00100000:2.0:1789760780.553956:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710663744917 time 710663484501 state 0x1 ref 1 00000100:00100000:2.0:1789760780.553984:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710663744917 time 710678590794 state 0x0 ref 1 00000100:00100000:2.0:1789760780.553998:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503806080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.554048:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77369344, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e60 00010000:00010000:2.0:1789760780.554056:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.554071:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.554075:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.554081:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.554094:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.559900:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986196 is committed 00002000:00010000:2.0:1789760780.560012:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77369344 -> 77373440 00002000:00010000:2.0:1789760780.560018:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151136 -> 151144 00010000:00010000:2.0:1789760780.560030:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.560041:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.560049:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.560056:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.560061:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.560075:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.560093:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.560100:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77369344->77373439] (req 77369344->77373439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.560193:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503806080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6193us (6418us total) trans 4294986196 rc 0/0 00000100:00100000:2.0:1789760780.560202:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18900 00000100:00100000:2.0:1789760780.560212:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710663744917 time 710678590794 state 0x2 ref 0 00000100:00100000:0.0:1789760780.562962:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.563053:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806208 00000100:00100000:1.0:1789760780.563063:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806208 00000100:00100000:1.0:1789760780.563118:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710663744917 time 710678590794 state 0x0 ref 1 00000100:00100000:1.0:1789760780.563133:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710678851210 time 710678590794 state 0x1 ref 1 00000100:00100000:1.0:1789760780.563165:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710678851210 time 710687771699 state 0x0 ref 1 00000100:00100000:1.0:1789760780.563177:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503806208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.563216:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77373440, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e68 00010000:00010000:1.0:1789760780.563225:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.563240:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.563244:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.563251:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.563265:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.567948:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986197 is committed 00002000:00010000:1.0:1789760780.568080:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77373440 -> 77377536 00002000:00010000:1.0:1789760780.568086:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151144 -> 151152 00010000:00010000:1.0:1789760780.568169:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.568181:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.568190:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.568198:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.568202:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.568211:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.568271:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.568280:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0c8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77373440->77377535] (req 77373440->77377535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.568538:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503806208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5358us (5580us total) trans 4294986197 rc 0/0 00000100:00100000:1.0:1789760780.568548:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18901 00000100:00100000:1.0:1789760780.568557:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710678851210 time 710687771699 state 0x2 ref 0 00000100:00100000:0.0:1789760780.577117:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.577206:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806336 00000100:00100000:3.0:1789760780.577215:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806336 00000100:00100000:3.0:1789760780.577281:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710678851210 time 710687771699 state 0x0 ref 1 00000100:00100000:3.0:1789760780.577292:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710688032115 time 710687771699 state 0x1 ref 1 00000100:00100000:3.0:1789760780.577327:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710688032115 time 710701933694 state 0x0 ref 1 00000100:00100000:3.0:1789760780.577340:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503806336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.577426:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77377536, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e70 00010000:00010000:3.0:1789760780.577437:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.577454:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.577459:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.577466:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.577481:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.584939:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986198 is committed 00002000:00010000:2.0:1789760780.585049:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77377536 -> 77381632 00002000:00010000:2.0:1789760780.585054:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151152 -> 151160 00010000:00010000:2.0:1789760780.585066:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.585079:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.585089:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.585100:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.585105:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.585113:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.585132:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.585141:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0c96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77377536->77381631] (req 77377536->77381631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.585554:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503806336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8048us (8278us total) trans 4294986198 rc 0/0 00000100:00100000:2.0:1789760780.585566:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18902 00000100:00100000:2.0:1789760780.585590:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710688032115 time 710701933694 state 0x2 ref 0 00000100:00100000:0.0:1789760780.589833:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.592836:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806464 00000100:00100000:2.0:1789760780.592851:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806464 00000100:00100000:2.0:1789760780.592911:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710688032115 time 710701933694 state 0x0 ref 1 00000100:00100000:2.0:1789760780.592926:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710702194110 time 710701933694 state 0x1 ref 1 00000100:00100000:2.0:1789760780.592964:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710702194110 time 710717570544 state 0x0 ref 1 00000100:00100000:2.0:1789760780.592981:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503806464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.593073:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77381632, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e78 00010000:00010000:2.0:1789760780.593082:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.593098:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.593102:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.593108:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.593121:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.604497:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986199 is committed 00002000:00010000:2.0:1789760780.604619:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77381632 -> 77385728 00002000:00010000:2.0:1789760780.604624:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151160 -> 151168 00010000:00010000:2.0:1789760780.604705:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.604721:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.604730:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.604737:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.604742:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.604751:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.604770:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.604777:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0c9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77381632->77385727] (req 77381632->77385727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.604869:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503806464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11888us (15043us total) trans 4294986199 rc 0/0 00000100:00100000:2.0:1789760780.604880:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18903 00000100:00100000:2.0:1789760780.604890:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710702194110 time 710717570544 state 0x2 ref 0 00000100:00100000:0.0:1789760780.611292:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.612756:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806592 00000100:00100000:2.0:1789760780.612769:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806592 00000100:00100000:2.0:1789760780.612829:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710702194110 time 710717570544 state 0x0 ref 1 00000100:00100000:2.0:1789760780.612844:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710717830960 time 710717570544 state 0x1 ref 1 00000100:00100000:2.0:1789760780.612878:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710717830960 time 710737484565 state 0x0 ref 1 00000100:00100000:2.0:1789760780.612890:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503806592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.612935:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77385728, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e80 00010000:00010000:2.0:1789760780.612945:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.612962:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.612967:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.612974:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.612989:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.620566:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986200 is committed 00002000:00010000:2.0:1789760780.620746:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77385728 -> 77389824 00002000:00010000:2.0:1789760780.620750:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151168 -> 151176 00010000:00010000:2.0:1789760780.620763:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.620775:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.620785:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.620792:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.620797:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.620807:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.620827:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.620836:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0ca4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77385728->77389823] (req 77385728->77389823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.620926:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503806592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8033us (9639us total) trans 4294986200 rc 0/0 00000100:00100000:2.0:1789760780.620936:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18904 00000100:00100000:2.0:1789760780.620947:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710717830960 time 710737484565 state 0x2 ref 0 00000100:00100000:0.0:1789760780.626555:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.626629:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806720 00000100:00100000:1.0:1789760780.626646:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806720 00000100:00100000:1.0:1789760780.626699:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710717830960 time 710737484565 state 0x0 ref 1 00000100:00100000:1.0:1789760780.626713:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710737744981 time 710737484565 state 0x1 ref 1 00000100:00100000:1.0:1789760780.626744:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710737744981 time 710751351320 state 0x0 ref 1 00000100:00100000:1.0:1789760780.626756:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503806720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.626807:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77389824, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e88 00010000:00010000:1.0:1789760780.626816:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.626831:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.626837:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.626842:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.626856:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.634488:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986201 is committed 00002000:00010000:1.0:1789760780.640022:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77389824 -> 77393920 00002000:00010000:1.0:1789760780.640033:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151176 -> 151184 00010000:00010000:1.0:1789760780.640047:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.640062:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.640072:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.640081:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.640086:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.640095:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.640116:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.640129:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0cab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77389824->77393919] (req 77389824->77393919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.640230:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503806720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13470us (13679us total) trans 4294986201 rc 0/0 00000100:00100000:1.0:1789760780.640240:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18905 00000100:00100000:1.0:1789760780.640325:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710737744981 time 710751351320 state 0x2 ref 0 00000100:00100000:0.0:1789760780.643612:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.643738:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806848 00000100:00100000:1.0:1789760780.643749:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806848 00000100:00100000:1.0:1789760780.643815:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710737744981 time 710751351320 state 0x0 ref 1 00000100:00100000:1.0:1789760780.643831:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710751611736 time 710751351320 state 0x1 ref 1 00000100:00100000:1.0:1789760780.643861:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710751611736 time 710768467957 state 0x0 ref 1 00000100:00100000:1.0:1789760780.643874:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503806848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.643919:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77393920, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e90 00010000:00010000:1.0:1789760780.643928:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.643943:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.643948:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.643954:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.643973:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.650903:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986202 is committed 00002000:00010000:1.0:1789760780.651106:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77393920 -> 77398016 00002000:00010000:1.0:1789760780.651111:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151184 -> 151192 00010000:00010000:1.0:1789760780.651125:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.651192:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.651202:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.651209:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.651213:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.651222:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.651242:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.651249:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77393920->77398015] (req 77393920->77398015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.651352:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503806848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7475us (7744us total) trans 4294986202 rc 0/0 00000100:00100000:1.0:1789760780.651361:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18906 00000100:00100000:1.0:1789760780.651371:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710751611736 time 710768467957 state 0x2 ref 0 00000100:00100000:0.0:1789760780.654588:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.655827:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503806976 00000100:00100000:2.0:1789760780.655840:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503806976 00000100:00100000:2.0:1789760780.655901:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710751611736 time 710768467957 state 0x0 ref 1 00000100:00100000:2.0:1789760780.655917:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710768728373 time 710768467957 state 0x1 ref 1 00000100:00100000:2.0:1789760780.655956:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710768728373 time 710780562318 state 0x0 ref 1 00000100:00100000:2.0:1789760780.656039:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503806976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.656149:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77398016, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24e98 00010000:00010000:2.0:1789760780.656158:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.656179:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.656183:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.656190:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.656204:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.661133:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986203 is committed 00002000:00010000:0.0:1789760780.661258:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77398016 -> 77402112 00002000:00010000:0.0:1789760780.661264:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151192 -> 151200 00010000:00010000:0.0:1789760780.661276:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.661290:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.661299:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.661307:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.661312:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.661321:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.661340:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.661347:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f0cb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77398016->77402111] (req 77398016->77402111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.662796:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503806976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6819us (8216us total) trans 4294986203 rc 0/0 00000100:00100000:0.0:1789760780.662809:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18907 00000100:00100000:0.0:1789760780.662819:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710768728373 time 710780562318 state 0x2 ref 0 00000100:00100000:0.0:1789760780.665677:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.665757:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503807104 00000100:00100000:2.0:1789760780.665767:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503807104 00000100:00100000:2.0:1789760780.665818:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710768728373 time 710780562318 state 0x0 ref 1 00000100:00100000:2.0:1789760780.665834:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710780822734 time 710780562318 state 0x1 ref 1 00000100:00100000:2.0:1789760780.665864:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710780822734 time 710790470753 state 0x0 ref 1 00000100:00100000:2.0:1789760780.665877:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503807104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.665921:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77402112, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ea0 00010000:00010000:2.0:1789760780.665932:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.665947:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.665952:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.665958:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.665978:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.672169:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986204 is committed 00002000:00010000:2.0:1789760780.672362:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77402112 -> 77406208 00002000:00010000:2.0:1789760780.672367:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151200 -> 151208 00010000:00010000:2.0:1789760780.672379:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.672417:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.672427:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.672434:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.672439:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.672448:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.672467:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.672474:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77402112->77406207] (req 77402112->77406207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.672564:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503807104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6686us (6892us total) trans 4294986204 rc 0/0 00000100:00100000:2.0:1789760780.672585:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18908 00000100:00100000:2.0:1789760780.672992:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710780822734 time 710790470753 state 0x2 ref 0 00000100:00100000:2.0:1789760780.677142:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.677261:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503807232 00000100:00100000:1.0:1789760780.677271:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503807232 00000100:00100000:1.0:1789760780.677539:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710780822734 time 710790470753 state 0x0 ref 1 00000100:00100000:1.0:1789760780.677554:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710790731169 time 710790470753 state 0x1 ref 1 00000100:00100000:1.0:1789760780.677588:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710790731169 time 710802182826 state 0x0 ref 1 00000100:00100000:1.0:1789760780.677601:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503807232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.677708:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77406208, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ea8 00010000:00010000:1.0:1789760780.677717:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.677733:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.677738:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.677744:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.677763:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.685129:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986205 is committed 00002000:00010000:1.0:1789760780.685330:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77406208 -> 77410304 00002000:00010000:1.0:1789760780.685336:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151208 -> 151216 00010000:00010000:1.0:1789760780.685350:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.685363:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.685372:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.685379:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.685385:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.685428:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.685447:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.685456:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77406208->77410303] (req 77406208->77410303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.687167:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503807232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9558us (10023us total) trans 4294986205 rc 0/0 00000100:00100000:1.0:1789760780.687181:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18909 00000100:00100000:1.0:1789760780.687192:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710790731169 time 710802182826 state 0x2 ref 0 00000100:00100000:0.0:1789760780.693765:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.693853:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503807360 00000100:00100000:1.0:1789760780.693861:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503807360 00000100:00100000:1.0:1789760780.693925:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710790731169 time 710802182826 state 0x0 ref 1 00000100:00100000:1.0:1789760780.693947:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710802443242 time 710802182826 state 0x1 ref 1 00000100:00100000:2.0:1789760780.694079:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710802443242 time 710818682138 state 0x0 ref 1 00000100:00100000:2.0:1789760780.694096:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503807360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.694143:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77410304, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24eb0 00010000:00010000:2.0:1789760780.694153:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694168:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.694172:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.694179:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694193:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.698471:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986206 is committed 00002000:00010000:2.0:1789760780.698740:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77410304 -> 77414400 00002000:00010000:2.0:1789760780.698745:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151216 -> 151224 00010000:00010000:2.0:1789760780.698759:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.698772:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.698782:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.698789:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.698794:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.698804:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.698826:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.698834:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77410304->77414399] (req 77410304->77414399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.698981:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503807360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4885us (5220us total) trans 4294986206 rc 0/0 00000100:00100000:2.0:1789760780.698991:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18910 00000100:00100000:2.0:1789760780.699002:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710802443242 time 710818682138 state 0x2 ref 0 00000100:00100000:0.0:1789760780.704740:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.704843:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503807488 00000100:00100000:1.0:1789760780.704852:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503807488 00000100:00100000:1.0:1789760780.704906:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710802443242 time 710818682138 state 0x0 ref 1 00000100:00100000:1.0:1789760780.704921:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710818942554 time 710818682138 state 0x1 ref 1 00000100:00100000:1.0:1789760780.704956:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710818942554 time 710829562081 state 0x0 ref 1 00000100:00100000:1.0:1789760780.704966:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503807488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.705017:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77414400, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24eb8 00010000:00010000:1.0:1789760780.705025:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.705041:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.705046:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.705053:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.705066:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.709684:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986207 is committed 00002000:00010000:1.0:1789760780.709840:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77414400 -> 77418496 00002000:00010000:1.0:1789760780.709846:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151224 -> 151232 00010000:00010000:1.0:1789760780.709859:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.709870:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.709878:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.709884:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.709889:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.709897:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.709917:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.709930:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77414400->77418495] (req 77414400->77418495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.711733:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503807488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6757us (6991us total) trans 4294986207 rc 0/0 00000100:00100000:1.0:1789760780.711748:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18911 00000100:00100000:1.0:1789760780.711760:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710818942554 time 710829562081 state 0x2 ref 0 00000100:00100000:0.0:1789760780.719904:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.722415:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503807616 00000100:00100000:1.0:1789760780.722430:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503807616 00000100:00100000:1.0:1789760780.722494:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710818942554 time 710829562081 state 0x0 ref 1 00000100:00100000:1.0:1789760780.722510:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710829822497 time 710829562081 state 0x1 ref 1 00000100:00100000:1.0:1789760780.722540:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710829822497 time 710847146903 state 0x0 ref 1 00000100:00100000:1.0:1789760780.722554:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503807616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.722614:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77418496, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ec0 00010000:00010000:1.0:1789760780.722623:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.722639:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.722643:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.722650:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.722668:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.734116:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986208 is committed 00002000:00010000:1.0:1789760780.734239:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77418496 -> 77422592 00002000:00010000:1.0:1789760780.734244:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151232 -> 151240 00010000:00010000:1.0:1789760780.734258:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734269:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734277:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734285:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.734289:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734357:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734379:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734404:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cdc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77418496->77422591] (req 77418496->77422591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.734556:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503807616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12000us (14656us total) trans 4294986208 rc 0/0 00000100:00100000:1.0:1789760780.734567:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18912 00000100:00100000:1.0:1789760780.734587:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710829822497 time 710847146903 state 0x2 ref 0 00000100:00100000:0.0:1789760780.737622:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.737765:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503807744 00000100:00100000:2.0:1789760780.737774:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503807744 00000100:00100000:2.0:1789760780.737826:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710829822497 time 710847146903 state 0x0 ref 1 00000100:00100000:2.0:1789760780.737840:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710847407319 time 710847146903 state 0x1 ref 1 00000100:00100000:2.0:1789760780.737901:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710847407319 time 710862507087 state 0x0 ref 1 00000100:00100000:2.0:1789760780.737914:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503807744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.737962:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77422592, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ec8 00010000:00010000:2.0:1789760780.737971:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.737994:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.738000:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.738007:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.738020:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.746326:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986209 is committed 00002000:00010000:2.0:1789760780.746544:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77422592 -> 77426688 00002000:00010000:2.0:1789760780.746549:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151240 -> 151248 00010000:00010000:2.0:1789760780.746562:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.746599:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.746608:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.746616:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.746621:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.746632:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.746651:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.746659:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0ce3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77422592->77426687] (req 77422592->77426687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.746743:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503807744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8825us (9124us total) trans 4294986209 rc 0/0 00000100:00100000:2.0:1789760780.746754:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18913 00000100:00100000:2.0:1789760780.746763:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710847407319 time 710862507087 state 0x2 ref 0 00000100:00100000:0.0:1789760780.750172:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.750791:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503807872 00000100:00100000:3.0:1789760780.750802:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503807872 00000100:00100000:3.0:1789760780.750861:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710847407319 time 710862507087 state 0x0 ref 1 00000100:00100000:3.0:1789760780.750875:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710862767503 time 710862507087 state 0x1 ref 1 00000100:00100000:3.0:1789760780.750909:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710862767503 time 710875515055 state 0x0 ref 1 00000100:00100000:3.0:1789760780.750920:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503807872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.750968:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77426688, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ed0 00010000:00010000:3.0:1789760780.750977:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.751002:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760780.751007:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.751014:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.751027:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.758449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986210 is committed 00002000:00010000:3.0:1789760780.758610:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77426688 -> 77430784 00002000:00010000:3.0:1789760780.758616:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151248 -> 151256 00010000:00010000:3.0:1789760780.758628:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.758641:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.758650:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.758657:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.758662:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.758672:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.758690:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.758704:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0cea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77426688->77430783] (req 77426688->77430783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.759527:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503807872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8006us (8761us total) trans 4294986210 rc 0/0 00000100:00100000:3.0:1789760780.760122:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18914 00000100:00100000:3.0:1789760780.760134:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710862767503 time 710875515055 state 0x2 ref 0 00000100:00100000:0.0:1789760780.765007:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.765100:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808000 00000100:00100000:1.0:1789760780.765110:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808000 00000100:00100000:1.0:1789760780.765172:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710862767503 time 710875515055 state 0x0 ref 1 00000100:00100000:1.0:1789760780.765188:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710875775471 time 710875515055 state 0x1 ref 1 00000100:00100000:1.0:1789760780.765221:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710875775471 time 710889827686 state 0x0 ref 1 00000100:00100000:1.0:1789760780.765234:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503808000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.765280:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77430784, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ed8 00010000:00010000:1.0:1789760780.765289:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.765304:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.765309:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.765315:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.765328:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.770606:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986211 is committed 00002000:00010000:1.0:1789760780.770782:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77430784 -> 77434880 00002000:00010000:1.0:1789760780.770788:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151256 -> 151264 00010000:00010000:1.0:1789760780.770803:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.770817:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.770826:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.770833:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.770838:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.770847:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.770869:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.770878:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0cf1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77430784->77434879] (req 77430784->77434879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.770969:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503808000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5733us (5967us total) trans 4294986211 rc 0/0 00000100:00100000:1.0:1789760780.770979:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18915 00000100:00100000:1.0:1789760780.770998:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710875775471 time 710889827686 state 0x2 ref 0 00000100:00100000:0.0:1789760780.773632:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.774616:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808128 00000100:00100000:2.0:1789760780.774627:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808128 00000100:00100000:2.0:1789760780.774687:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710875775471 time 710889827686 state 0x0 ref 1 00000100:00100000:2.0:1789760780.774702:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710890088102 time 710889827686 state 0x1 ref 1 00000100:00100000:2.0:1789760780.774731:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710890088102 time 710899338269 state 0x0 ref 1 00000100:00100000:2.0:1789760780.774743:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503808128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.774786:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77434880, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ee0 00010000:00010000:2.0:1789760780.774795:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.774811:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.774815:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.774822:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.774836:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.782695:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986212 is committed 00002000:00010000:2.0:1789760780.783056:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77434880 -> 77438976 00002000:00010000:2.0:1789760780.783062:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151264 -> 151272 00010000:00010000:2.0:1789760780.783075:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.783087:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.783100:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.783107:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.783112:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.783120:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.783139:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.783146:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cf8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77434880->77438975] (req 77434880->77438975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.783325:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503808128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8577us (9696us total) trans 4294986212 rc 0/0 00000100:00100000:2.0:1789760780.783337:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18916 00000100:00100000:2.0:1789760780.783347:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710890088102 time 710899338269 state 0x2 ref 0 00000100:00100000:0.0:1789760780.788834:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.789038:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808256 00000100:00100000:2.0:1789760780.789048:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808256 00000100:00100000:2.0:1789760780.789099:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710890088102 time 710899338269 state 0x0 ref 1 00000100:00100000:2.0:1789760780.789114:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710899598685 time 710899338269 state 0x1 ref 1 00000100:00100000:2.0:1789760780.789184:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710899598685 time 710913789163 state 0x0 ref 1 00000100:00100000:2.0:1789760780.789197:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503808256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.789245:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77438976, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x24ee8 00010000:00010000:2.0:1789760780.789255:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.789269:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.789274:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.789281:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.789295:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.794224:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986213 is committed 00002000:00010000:2.0:1789760780.794347:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77438976 -> 77443072 00002000:00010000:2.0:1789760780.794352:0:15091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:2.0:1789760780.794355:0:15091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00002000:00010000:2.0:1789760780.794357:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151272 -> 151280 00010000:00010000:2.0:1789760780.794370:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.794383:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.794518:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.794527:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.794532:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.794541:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.794563:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.794571:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0cff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77438976->77443071] (req 77438976->77443071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.794720:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503808256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5520us (5891us total) trans 4294986213 rc 0/0 00000100:00100000:2.0:1789760780.794731:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18917 00000100:00100000:2.0:1789760780.794742:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710899598685 time 710913789163 state 0x2 ref 0 00000100:00100000:0.0:1789760780.800544:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.800717:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808384 00000100:00100000:2.0:1789760780.800729:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808384 00000100:00100000:2.0:1789760780.800785:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710899598685 time 710913789163 state 0x0 ref 1 00000100:00100000:2.0:1789760780.800800:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710914049579 time 710913789163 state 0x1 ref 1 00000100:00100000:2.0:1789760780.801025:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710914049579 time 710925630081 state 0x0 ref 1 00000100:00100000:2.0:1789760780.801042:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503808384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.801090:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77443072, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24ef0 00010000:00010000:2.0:1789760780.801099:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.801114:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.801119:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.801125:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.801138:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.805616:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986214 is committed 00002000:00010000:2.0:1789760780.805730:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77443072 -> 77447168 00002000:00010000:2.0:1789760780.805735:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151280 -> 151288 00010000:00010000:2.0:1789760780.805747:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.805759:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.805768:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.805776:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.805781:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.805791:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.805811:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.805818:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f0d06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77443072->77447167] (req 77443072->77447167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.806260:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503808384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4866us (5370us total) trans 4294986214 rc 0/0 00000100:00100000:2.0:1789760780.806271:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18918 00000100:00100000:2.0:1789760780.806281:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710914049579 time 710925630081 state 0x2 ref 0 00000100:00100000:0.0:1789760780.808832:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.809126:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808512 00000100:00100000:1.0:1789760780.809137:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808512 00000100:00100000:1.0:1789760780.809190:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710914049579 time 710925630081 state 0x0 ref 1 00000100:00100000:1.0:1789760780.809206:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710925890497 time 710925630081 state 0x1 ref 1 00000100:00100000:1.0:1789760780.809270:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710925890497 time 710933874871 state 0x0 ref 1 00000100:00100000:1.0:1789760780.809283:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503808512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.809344:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77447168, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24ef8 00010000:00010000:1.0:1789760780.809355:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.809371:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.809376:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.809382:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.809428:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.814000:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986215 is committed 00002000:00010000:0.0:1789760780.814190:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77447168 -> 77451264 00002000:00010000:0.0:1789760780.814194:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151288 -> 151296 00010000:00010000:0.0:1789760780.814207:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.814220:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.814229:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.814235:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.814238:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.814245:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.814260:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.814269:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f0d0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77447168->77451263] (req 77447168->77451263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.814476:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503808512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5190us (5648us total) trans 4294986215 rc 0/0 00000100:00100000:0.0:1789760780.814487:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18919 00000100:00100000:0.0:1789760780.814496:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710925890497 time 710933874871 state 0x2 ref 0 00010000:00010000:3.0:1789760780.818786:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760780.819631:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.821228:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808640 00000100:00100000:1.0:1789760780.821241:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808640 00000100:00100000:1.0:1789760780.821361:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710925890497 time 710933874871 state 0x0 ref 1 00000100:00100000:1.0:1789760780.821376:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710934135287 time 710933874871 state 0x1 ref 1 00000100:00100000:1.0:1789760780.821743:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710934135287 time 710946347922 state 0x0 ref 1 00000100:00100000:1.0:1789760780.821767:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503808640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.821817:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77451264, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f00 00010000:00010000:1.0:1789760780.821826:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.821841:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.821846:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.821852:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.821866:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.826856:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986216 is committed 00002000:00010000:1.0:1789760780.827543:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77451264 -> 77455360 00002000:00010000:1.0:1789760780.827549:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151296 -> 151304 00010000:00010000:1.0:1789760780.827561:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.827584:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.827599:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.827717:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.827723:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.827734:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.827755:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.827762:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77451264->77455359] (req 77451264->77455359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.827907:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503808640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6143us (8278us total) trans 4294986216 rc 0/0 00000100:00100000:1.0:1789760780.827918:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18920 00000100:00100000:1.0:1789760780.827927:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710934135287 time 710946347922 state 0x2 ref 0 00000100:00100000:2.0:1789760780.839222:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.839759:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808768 00000100:00100000:1.0:1789760780.839772:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808768 00000100:00100000:1.0:1789760780.839840:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710934135287 time 710946347922 state 0x0 ref 1 00000100:00100000:1.0:1789760780.839857:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710946608338 time 710946347922 state 0x1 ref 1 00000100:00100000:1.0:1789760780.840208:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710946608338 time 710964813054 state 0x0 ref 1 00000100:00100000:1.0:1789760780.840225:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503808768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.840278:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77455360, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f08 00010000:00010000:1.0:1789760780.840288:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.840305:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760780.840310:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.840316:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.840329:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.849853:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986217 is committed 00002000:00010000:1.0:1789760780.850045:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77455360 -> 77459456 00002000:00010000:1.0:1789760780.850051:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151304 -> 151312 00010000:00010000:1.0:1789760780.850065:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.850078:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.850088:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.850102:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.850107:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.850115:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.850135:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.850144:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77455360->77459455] (req 77455360->77459455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.850240:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503808768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10013us (11025us total) trans 4294986217 rc 0/0 00000100:00100000:1.0:1789760780.850251:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18921 00000100:00100000:1.0:1789760780.850260:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710946608338 time 710964813054 state 0x2 ref 0 00000100:00100000:0.0:1789760780.861375:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.861547:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503808896 00000100:00100000:2.0:1789760780.861558:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503808896 00000100:00100000:2.0:1789760780.861685:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710946608338 time 710964813054 state 0x0 ref 1 00000100:00100000:2.0:1789760780.861701:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710965073470 time 710964813054 state 0x1 ref 1 00000100:00100000:2.0:1789760780.861784:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710965073470 time 710986389413 state 0x0 ref 1 00000100:00100000:2.0:1789760780.861798:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503808896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.861851:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77459456, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f10 00010000:00010000:2.0:1789760780.861861:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.861877:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.861882:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.861888:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.861901:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.869721:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986218 is committed 00002000:00010000:2.0:1789760780.870306:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77459456 -> 77463552 00002000:00010000:2.0:1789760780.870312:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151312 -> 151320 00010000:00010000:2.0:1789760780.870325:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870339:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870349:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870357:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.870361:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870375:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.871244:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.871261:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77459456->77463551] (req 77459456->77463551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.871478:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503808896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9678us (10110us total) trans 4294986218 rc 0/0 00000100:00100000:2.0:1789760780.871489:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18922 00000100:00100000:2.0:1789760780.871499:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710965073470 time 710986389413 state 0x2 ref 0 00000100:00100000:0.0:1789760780.874958:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.875599:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809024 00000100:00100000:2.0:1789760780.875791:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809024 00000100:00100000:2.0:1789760780.875852:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710965073470 time 710986389413 state 0x0 ref 1 00000100:00100000:2.0:1789760780.875867:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710986649829 time 710986389413 state 0x1 ref 1 00000100:00100000:2.0:1789760780.875900:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710986649829 time 711000506652 state 0x0 ref 1 00000100:00100000:2.0:1789760780.875914:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503809024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.875960:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77463552, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f18 00010000:00010000:2.0:1789760780.875969:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.875986:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760780.875990:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.875996:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.876010:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.882364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986219 is committed 00002000:00010000:2.0:1789760780.882626:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77463552 -> 77467648 00002000:00010000:2.0:1789760780.882632:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151320 -> 151328 00010000:00010000:2.0:1789760780.882644:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.882656:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.882664:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.882671:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.882676:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.882683:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.882709:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.882716:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0d29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77463552->77467647] (req 77463552->77467647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.882813:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503809024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6895us (7858us total) trans 4294986219 rc 0/0 00000100:00100000:2.0:1789760780.882824:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18923 00000100:00100000:2.0:1789760780.882834:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710986649829 time 711000506652 state 0x2 ref 0 00000100:00100000:0.0:1789760780.889461:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.889584:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809152 00000100:00100000:1.0:1789760780.889595:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809152 00000100:00100000:1.0:1789760780.889779:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 710986649829 time 711000506652 state 0x0 ref 1 00000100:00100000:1.0:1789760780.889797:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711000767068 time 711000506652 state 0x1 ref 1 00000100:00100000:1.0:1789760780.889831:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711000767068 time 711014437234 state 0x0 ref 1 00000100:00100000:1.0:1789760780.889847:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503809152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.890022:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77467648, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f20 00010000:00010000:1.0:1789760780.890032:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.890050:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760780.890055:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.890061:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.890075:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.897987:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986220 is committed 00002000:00010000:1.0:1789760780.898250:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77467648 -> 77471744 00002000:00010000:1.0:1789760780.898256:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151328 -> 151336 00010000:00010000:1.0:1789760780.898269:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.898282:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.898290:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.898307:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.898313:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.898322:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.898341:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.898350:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77467648->77471743] (req 77467648->77471743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.898482:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503809152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8633us (9025us total) trans 4294986220 rc 0/0 00000100:00100000:1.0:1789760780.898492:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18924 00000100:00100000:1.0:1789760780.898502:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711000767068 time 711014437234 state 0x2 ref 0 00000100:00100000:0.0:1789760780.902245:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.902588:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809280 00000100:00100000:1.0:1789760780.902598:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809280 00000100:00100000:0.0:1789760780.902703:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711000767068 time 711014437234 state 0x0 ref 1 00000100:00100000:0.0:1789760780.902718:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711014697650 time 711014437234 state 0x1 ref 1 00000100:00100000:0.0:1789760780.902748:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711014697650 time 711027355018 state 0x0 ref 1 00000100:00100000:0.0:1789760780.902761:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503809280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760780.902807:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77471744, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f28 00010000:00010000:0.0:1789760780.902816:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902830:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760780.902836:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.902842:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902854:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.909216:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986221 is committed 00002000:00010000:0.0:1789760780.909455:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77471744 -> 77475840 00002000:00010000:0.0:1789760780.909460:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151336 -> 151344 00010000:00010000:0.0:1789760780.909472:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.909485:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.909494:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.909523:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.909527:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.909536:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.909555:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.909569:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0d37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77471744->77475839] (req 77471744->77475839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.909679:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503809280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6915us (7437us total) trans 4294986221 rc 0/0 00000100:00100000:0.0:1789760780.909689:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18925 00000100:00100000:0.0:1789760780.909699:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711014697650 time 711027355018 state 0x2 ref 0 00000100:00100000:0.0:1789760780.920121:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.920224:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809408 00000100:00100000:2.0:1789760780.920233:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809408 00000100:00100000:2.0:1789760780.920289:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711014697650 time 711027355018 state 0x0 ref 1 00000100:00100000:2.0:1789760780.920303:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711027615434 time 711027355018 state 0x1 ref 1 00000100:00100000:2.0:1789760780.920335:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711027615434 time 711044942419 state 0x0 ref 1 00000100:00100000:2.0:1789760780.920348:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503809408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.920429:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77475840, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f30 00010000:00010000:2.0:1789760780.920438:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.920455:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760780.920461:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.920467:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.920710:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.927664:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986222 is committed 00002000:00010000:2.0:1789760780.927788:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77475840 -> 77479936 00002000:00010000:2.0:1789760780.927794:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151344 -> 151352 00010000:00010000:2.0:1789760780.927808:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.927822:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.927831:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.927838:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.927842:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.927851:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.927873:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.927897:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0d3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77475840->77479935] (req 77475840->77479935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.928001:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503809408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7650us (7884us total) trans 4294986222 rc 0/0 00000100:00100000:2.0:1789760780.928011:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18926 00000100:00100000:2.0:1789760780.928021:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711027615434 time 711044942419 state 0x2 ref 0 00000100:00100000:2.0:1789760780.936536:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760780.936887:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809536 00000100:00100000:0.0:1789760780.936897:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809536 00000100:00100000:0.0:1789760780.936951:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711027615434 time 711044942419 state 0x0 ref 1 00000100:00100000:0.0:1789760780.936967:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711045202835 time 711044942419 state 0x1 ref 1 00000100:00100000:0.0:1789760780.936998:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711045202835 time 711061604434 state 0x0 ref 1 00000100:00100000:0.0:1789760780.937010:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503809536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760780.937067:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77479936, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f38 00010000:00010000:0.0:1789760780.937077:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.937092:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760780.937096:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.937102:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.937116:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.944004:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986223 is committed 00002000:00010000:0.0:1789760780.944143:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77479936 -> 77484032 00002000:00010000:0.0:1789760780.944149:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151352 -> 151360 00010000:00010000:0.0:1789760780.944163:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.944175:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.944184:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.944192:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760780.944197:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.944206:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.944224:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.944231:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77479936->77484031] (req 77479936->77484031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.944324:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503809536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7312us (7794us total) trans 4294986223 rc 0/0 00000100:00100000:0.0:1789760780.944334:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18927 00000100:00100000:0.0:1789760780.944344:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711045202835 time 711061604434 state 0x2 ref 0 00000100:00100000:0.0:1789760780.955017:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760780.955078:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809664 00000100:00100000:1.0:1789760780.955087:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809664 00000100:00100000:1.0:1789760780.955142:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711045202835 time 711061604434 state 0x0 ref 1 00000100:00100000:1.0:1789760780.955158:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711061864850 time 711061604434 state 0x1 ref 1 00000100:00100000:1.0:1789760780.955186:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711061864850 time 711079793150 state 0x0 ref 1 00000100:00100000:1.0:1789760780.955198:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503809664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760780.955256:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77484032, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f40 00010000:00010000:1.0:1789760780.955265:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.955280:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760780.955284:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.955289:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.955303:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.962013:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986224 is committed 00002000:00010000:1.0:1789760780.962172:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77484032 -> 77488128 00002000:00010000:1.0:1789760780.962177:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151360 -> 151368 00010000:00010000:1.0:1789760780.962189:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.962201:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.962209:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.962217:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760780.962221:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.962234:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.962253:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.962261:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77484032->77488127] (req 77484032->77488127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.962358:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503809664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7158us (7356us total) trans 4294986224 rc 0/0 00000100:00100000:1.0:1789760780.962366:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18928 00000100:00100000:1.0:1789760780.962375:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711061864850 time 711079793150 state 0x2 ref 0 00000100:00100000:2.0:1789760780.967356:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760780.967730:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809792 00000100:00100000:0.0:1789760780.967743:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809792 00000100:00100000:0.0:1789760780.967814:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711061864850 time 711079793150 state 0x0 ref 1 00000100:00100000:0.0:1789760780.967839:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711080053566 time 711079793150 state 0x1 ref 1 00000100:00100000:0.0:1789760780.967878:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711080053566 time 711092483038 state 0x0 ref 1 00000100:00100000:0.0:1789760780.967894:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503809792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760780.967947:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77488128, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f48 00010000:00010000:0.0:1789760780.967956:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.967971:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760780.967975:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.967986:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.968001:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.977640:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986225 is committed 00002000:00010000:2.0:1789760780.977767:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77488128 -> 77492224 00002000:00010000:2.0:1789760780.977771:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151368 -> 151376 00010000:00010000:2.0:1789760780.977782:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.977791:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.977797:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.977802:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760780.977806:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.977813:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.977829:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.977833:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f0d53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77488128->77492223] (req 77488128->77492223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.977912:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503809792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10017us (10561us total) trans 4294986225 rc 0/0 00000100:00100000:2.0:1789760780.977923:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18929 00000100:00100000:2.0:1789760780.977932:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711080053566 time 711092483038 state 0x2 ref 0 00000100:00100000:2.0:1789760780.981678:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760780.981845:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503809920 00000100:00100000:3.0:1789760780.981854:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503809920 00000100:00100000:3.0:1789760780.981908:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711080053566 time 711092483038 state 0x0 ref 1 00000100:00100000:3.0:1789760780.983444:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711092743454 time 711092483038 state 0x1 ref 1 00000100:00100000:3.0:1789760780.983494:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711092743454 time 711108099966 state 0x0 ref 1 00000100:00100000:3.0:1789760780.983516:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503809920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760780.983566:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77492224, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f50 00010000:00010000:3.0:1789760780.983583:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.983598:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760780.983603:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.983609:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.983629:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.990364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986226 is committed 00002000:00010000:3.0:1789760780.990771:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77492224 -> 77496320 00002000:00010000:3.0:1789760780.990777:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151376 -> 151384 00010000:00010000:3.0:1789760780.990791:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.990804:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.990814:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.990821:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760780.990826:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.990841:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.990861:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.990869:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77492224->77496319] (req 77492224->77496319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.990965:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503809920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7452us (9409us total) trans 4294986226 rc 0/0 00000100:00100000:3.0:1789760780.990974:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18930 00000100:00100000:3.0:1789760780.990985:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711092743454 time 711108099966 state 0x2 ref 0 00000100:00100000:0.0:1789760780.995984:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760780.996186:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810048 00000100:00100000:2.0:1789760780.996196:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810048 00000100:00100000:2.0:1789760780.996248:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711092743454 time 711108099966 state 0x0 ref 1 00000100:00100000:2.0:1789760780.996264:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711108360382 time 711108099966 state 0x1 ref 1 00000100:00100000:2.0:1789760780.996295:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711108360382 time 711120901827 state 0x0 ref 1 00000100:00100000:2.0:1789760780.996308:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760780.996353:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77496320, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f58 00010000:00010000:2.0:1789760780.996363:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.996378:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760780.996383:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.996685:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.996711:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.002039:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986227 is committed 00002000:00010000:2.0:1789760781.002547:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77496320 -> 77500416 00002000:00010000:2.0:1789760781.002553:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151384 -> 151392 00010000:00010000:2.0:1789760781.002568:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.002588:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.002596:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.002603:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.002608:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.002617:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.002638:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.002645:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77496320->77500415] (req 77496320->77500415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.002745:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6434us (6765us total) trans 4294986227 rc 0/0 00000100:00100000:2.0:1789760781.002755:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18931 00000100:00100000:2.0:1789760781.002765:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711108360382 time 711120901827 state 0x2 ref 0 00000100:00100000:2.0:1789760781.009982:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.010217:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810176 00000100:00100000:1.0:1789760781.010228:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810176 00000100:00100000:1.0:1789760781.010284:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711108360382 time 711120901827 state 0x0 ref 1 00000100:00100000:1.0:1789760781.010300:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711121162243 time 711120901827 state 0x1 ref 1 00000100:00100000:1.0:1789760781.010374:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711121162243 time 711134979567 state 0x0 ref 1 00000100:00100000:1.0:1789760781.010387:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503810176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.010465:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77500416, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f60 00010000:00010000:1.0:1789760781.010475:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.010504:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.010508:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.010514:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.010529:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.017273:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986228 is committed 00002000:00010000:1.0:1789760781.017418:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77500416 -> 77504512 00002000:00010000:1.0:1789760781.017424:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151392 -> 151400 00010000:00010000:1.0:1789760781.017443:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.017454:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.017463:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.017469:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.017474:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.017482:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.017504:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.017511:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77500416->77504511] (req 77500416->77504511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.018211:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503810176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7820us (8235us total) trans 4294986228 rc 0/0 00000100:00100000:1.0:1789760781.018223:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18932 00000100:00100000:1.0:1789760781.018233:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711121162243 time 711134979567 state 0x2 ref 0 00000100:00100000:0.0:1789760781.021537:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.021730:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810304 00000100:00100000:2.0:1789760781.021740:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810304 00000100:00100000:2.0:1789760781.021811:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711121162243 time 711134979567 state 0x0 ref 1 00000100:00100000:2.0:1789760781.021829:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711135239983 time 711134979567 state 0x1 ref 1 00000100:00100000:2.0:1789760781.021864:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711135239983 time 711146470125 state 0x0 ref 1 00000100:00100000:2.0:1789760781.021878:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc17050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.021925:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77504512, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f68 00010000:00010000:2.0:1789760781.021935:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.021956:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.021960:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.021966:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.021981:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.027631:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986229 is committed 00002000:00010000:2.0:1789760781.027754:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77504512 -> 77508608 00002000:00010000:2.0:1789760781.027759:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151400 -> 151408 00010000:00010000:2.0:1789760781.027772:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.027785:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.027794:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.027802:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.027806:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.027815:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.027840:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.027847:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77504512->77508607] (req 77504512->77508607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.027938:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc17050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6058us (6407us total) trans 4294986229 rc 0/0 00000100:00100000:2.0:1789760781.027948:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18933 00000100:00100000:2.0:1789760781.027958:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711135239983 time 711146470125 state 0x2 ref 0 00000100:00100000:0.0:1789760781.033006:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.036972:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810432 00000100:00100000:3.0:1789760781.036987:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810432 00000100:00100000:3.0:1789760781.037053:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711135239983 time 711146470125 state 0x0 ref 1 00000100:00100000:3.0:1789760781.037069:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711146730541 time 711146470125 state 0x1 ref 1 00000100:00100000:3.0:1789760781.037104:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711146730541 time 711161710678 state 0x0 ref 1 00000100:00100000:3.0:1789760781.037117:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503810432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.037165:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77508608, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f70 00010000:00010000:3.0:1789760781.037175:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.037191:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.037196:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.037201:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.037215:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.046514:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986230 is committed 00002000:00010000:3.0:1789760781.046653:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77508608 -> 77512704 00002000:00010000:3.0:1789760781.046658:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151408 -> 151416 00010000:00010000:3.0:1789760781.046672:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.046685:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.046694:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.046701:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.046706:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.046715:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.046737:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.046745:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77508608->77512703] (req 77508608->77512703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.047012:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503810432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9891us (14140us total) trans 4294986230 rc 0/0 00000100:00100000:3.0:1789760781.047023:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18934 00000100:00100000:3.0:1789760781.047032:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711146730541 time 711161710678 state 0x2 ref 0 00000100:00100000:0.0:1789760781.057512:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.057614:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810560 00000100:00100000:2.0:1789760781.057622:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810560 00000100:00100000:2.0:1789760781.057674:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711146730541 time 711161710678 state 0x0 ref 1 00000100:00100000:2.0:1789760781.057689:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711161971094 time 711161710678 state 0x1 ref 1 00000100:00100000:2.0:1789760781.057730:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711161971094 time 711182336482 state 0x0 ref 1 00000100:00100000:2.0:1789760781.057743:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503810560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.057791:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77512704, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f78 00010000:00010000:2.0:1789760781.057799:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.057814:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.057819:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.057825:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.057838:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.066223:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986231 is committed 00002000:00010000:2.0:1789760781.067231:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77512704 -> 77516800 00002000:00010000:2.0:1789760781.067238:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151416 -> 151424 00010000:00010000:2.0:1789760781.067251:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.067264:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.067273:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.067281:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.067286:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.067306:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.067330:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.067343:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77512704->77516799] (req 77512704->77516799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.068709:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503810560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10958us (11198us total) trans 4294986231 rc 0/0 00000100:00100000:2.0:1789760781.068724:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18935 00000100:00100000:2.0:1789760781.068734:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711161971094 time 711182336482 state 0x2 ref 0 00000100:00100000:0.0:1789760781.071809:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.071897:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810688 00000100:00100000:1.0:1789760781.071906:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810688 00000100:00100000:1.0:1789760781.071960:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711161971094 time 711182336482 state 0x0 ref 1 00000100:00100000:1.0:1789760781.071976:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711182596898 time 711182336482 state 0x1 ref 1 00000100:00100000:1.0:1789760781.072002:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711182596898 time 711196608831 state 0x0 ref 1 00000100:00100000:1.0:1789760781.072014:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.072073:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77516800, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f80 00010000:00010000:1.0:1789760781.072082:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.072105:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.072109:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.072116:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.072130:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.082430:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986232 is committed 00002000:00010000:1.0:1789760781.082533:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77516800 -> 77520896 00002000:00010000:1.0:1789760781.082539:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151424 -> 151432 00010000:00010000:1.0:1789760781.082553:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.082566:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.082590:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.082597:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.082601:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.082609:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.082634:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.082642:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0d84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77516800->77520895] (req 77516800->77520895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.082731:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10714us (10928us total) trans 4294986232 rc 0/0 00000100:00100000:1.0:1789760781.082740:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18936 00000100:00100000:1.0:1789760781.082750:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711182596898 time 711196608831 state 0x2 ref 0 00000100:00100000:0.0:1789760781.091237:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.092549:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810816 00000100:00100000:2.0:1789760781.092562:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810816 00000100:00100000:2.0:1789760781.092695:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711182596898 time 711196608831 state 0x0 ref 1 00000100:00100000:2.0:1789760781.092780:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711196869247 time 711196608831 state 0x1 ref 1 00000100:00100000:2.0:1789760781.092814:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711196869247 time 711217420891 state 0x0 ref 1 00000100:00100000:2.0:1789760781.092827:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.093010:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77520896, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f88 00010000:00010000:2.0:1789760781.093021:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.093105:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.093111:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.093118:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.093461:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.099904:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986233 is committed 00002000:00010000:2.0:1789760781.100022:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77520896 -> 77524992 00002000:00010000:2.0:1789760781.100028:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151432 -> 151440 00010000:00010000:2.0:1789760781.100040:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.100053:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.100062:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.100069:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.100074:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.100083:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.100103:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.100110:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77520896->77524991] (req 77520896->77524991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.100202:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503810816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7372us (9012us total) trans 4294986233 rc 0/0 00000100:00100000:2.0:1789760781.100213:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18937 00000100:00100000:2.0:1789760781.100224:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711196869247 time 711217420891 state 0x2 ref 0 00000100:00100000:0.0:1789760781.105855:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.106007:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503810944 00000100:00100000:3.0:1789760781.106017:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503810944 00000100:00100000:3.0:1789760781.106083:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711196869247 time 711217420891 state 0x0 ref 1 00000100:00100000:3.0:1789760781.106098:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711217681307 time 711217420891 state 0x1 ref 1 00000100:00100000:3.0:1789760781.106131:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711217681307 time 711230737645 state 0x0 ref 1 00000100:00100000:3.0:1789760781.106144:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503810944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.106188:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77524992, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f90 00010000:00010000:3.0:1789760781.106198:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106214:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.106219:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.106226:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106240:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.113496:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986234 is committed 00002000:00010000:3.0:1789760781.113692:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77524992 -> 77529088 00002000:00010000:3.0:1789760781.113698:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151440 -> 151448 00010000:00010000:3.0:1789760781.113711:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.113723:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.113732:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.113740:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.113744:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.113753:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.113771:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.113786:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0d92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77524992->77529087] (req 77524992->77529087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.113923:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503810944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7777us (8073us total) trans 4294986234 rc 0/0 00000100:00100000:3.0:1789760781.113934:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18938 00000100:00100000:3.0:1789760781.113944:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711217681307 time 711230737645 state 0x2 ref 0 00000100:00100000:0.0:1789760781.122665:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.122744:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811072 00000100:00100000:2.0:1789760781.122753:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811072 00000100:00100000:2.0:1789760781.122803:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711217681307 time 711230737645 state 0x0 ref 1 00000100:00100000:2.0:1789760781.122817:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711230998061 time 711230737645 state 0x1 ref 1 00000100:00100000:2.0:1789760781.122846:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711230998061 time 711247452470 state 0x0 ref 1 00000100:00100000:2.0:1789760781.122857:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.122901:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77529088, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24f98 00010000:00010000:2.0:1789760781.122910:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.122925:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.122929:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.122935:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.122948:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.128902:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986235 is committed 00002000:00010000:2.0:1789760781.129018:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77529088 -> 77533184 00002000:00010000:2.0:1789760781.129023:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151448 -> 151456 00010000:00010000:2.0:1789760781.129036:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.129048:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.129057:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.129064:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.129706:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.129721:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.129741:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.129749:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0d99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77529088->77533183] (req 77529088->77533183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.129843:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6982us (7182us total) trans 4294986235 rc 0/0 00000100:00100000:2.0:1789760781.129854:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18939 00000100:00100000:2.0:1789760781.129863:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711230998061 time 711247452470 state 0x2 ref 0 00000100:00100000:0.0:1789760781.144103:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.144196:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811200 00000100:00100000:1.0:1789760781.144206:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811200 00000100:00100000:1.0:1789760781.144261:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711230998061 time 711247452470 state 0x0 ref 1 00000100:00100000:1.0:1789760781.144276:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711247712886 time 711247452470 state 0x1 ref 1 00000100:00100000:1.0:1789760781.144313:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711247712886 time 711268919851 state 0x0 ref 1 00000100:00100000:1.0:1789760781.144326:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503811200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.144374:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77533184, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fa0 00010000:00010000:1.0:1789760781.144384:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.144442:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.144448:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.144453:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.144467:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.152249:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986236 is committed 00002000:00010000:1.0:1789760781.152560:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77533184 -> 77537280 00002000:00010000:1.0:1789760781.152565:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151456 -> 151464 00010000:00010000:1.0:1789760781.152587:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.152600:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.152609:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.152681:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.152686:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.152695:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.152714:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.152728:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0da0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77533184->77537279] (req 77533184->77537279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.152900:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503811200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8571us (8802us total) trans 4294986236 rc 0/0 00000100:00100000:1.0:1789760781.152910:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18940 00000100:00100000:1.0:1789760781.152918:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711247712886 time 711268919851 state 0x2 ref 0 00000100:00100000:0.0:1789760781.162768:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.163989:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811328 00000100:00100000:2.0:1789760781.164002:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811328 00000100:00100000:2.0:1789760781.164060:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711247712886 time 711268919851 state 0x0 ref 1 00000100:00100000:2.0:1789760781.164173:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711269180267 time 711268919851 state 0x1 ref 1 00000100:00100000:2.0:1789760781.164252:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711269180267 time 711288858769 state 0x0 ref 1 00000100:00100000:2.0:1789760781.164266:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503811328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.164325:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77537280, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fa8 00010000:00010000:2.0:1789760781.164333:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.164349:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.164353:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.164359:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.164373:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.174486:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986237 is committed 00002000:00010000:2.0:1789760781.174870:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77537280 -> 77541376 00002000:00010000:2.0:1789760781.174876:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151464 -> 151472 00010000:00010000:2.0:1789760781.174891:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174904:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174913:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174920:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.174924:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174932:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174950:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.174957:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0da7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77537280->77541375] (req 77537280->77541375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.175100:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503811328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10831us (12335us total) trans 4294986237 rc 0/0 00000100:00100000:2.0:1789760781.175114:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18941 00000100:00100000:2.0:1789760781.175125:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711269180267 time 711288858769 state 0x2 ref 0 00000100:00100000:0.0:1789760781.181737:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.181996:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811456 00000100:00100000:3.0:1789760781.182007:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811456 00000100:00100000:3.0:1789760781.182065:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711269180267 time 711288858769 state 0x0 ref 1 00000100:00100000:3.0:1789760781.182091:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711289119185 time 711288858769 state 0x1 ref 1 00000100:00100000:3.0:1789760781.182152:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711289119185 time 711306758721 state 0x0 ref 1 00000100:00100000:3.0:1789760781.182165:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.182209:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77541376, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fb0 00010000:00010000:3.0:1789760781.182218:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.182235:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.182239:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.182245:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.182263:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.189058:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986238 is committed 00002000:00010000:3.0:1789760781.190546:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77541376 -> 77545472 00002000:00010000:3.0:1789760781.190553:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151472 -> 151480 00010000:00010000:3.0:1789760781.190566:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.190813:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.190824:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.190830:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.190835:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.190843:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.190861:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.190868:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0dae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77541376->77545471] (req 77541376->77545471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.191009:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8841us (9435us total) trans 4294986238 rc 0/0 00000100:00100000:3.0:1789760781.191020:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18942 00000100:00100000:3.0:1789760781.191030:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711289119185 time 711306758721 state 0x2 ref 0 00000100:00100000:2.0:1789760781.195061:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.195137:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811584 00000100:00100000:0.0:1789760781.195146:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811584 00000100:00100000:0.0:1789760781.195199:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711289119185 time 711306758721 state 0x0 ref 1 00000100:00100000:0.0:1789760781.195214:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711307019137 time 711306758721 state 0x1 ref 1 00000100:00100000:0.0:1789760781.195246:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711307019137 time 711319853027 state 0x0 ref 1 00000100:00100000:0.0:1789760781.195258:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.195314:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77545472, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fb8 00010000:00010000:0.0:1789760781.195324:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.195340:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760781.195344:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.195350:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.195364:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.201111:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986239 is committed 00002000:00010000:0.0:1789760781.201717:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77545472 -> 77549568 00002000:00010000:0.0:1789760781.201723:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151480 -> 151488 00010000:00010000:0.0:1789760781.201736:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.201748:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.201757:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.201771:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.201776:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.201784:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.201804:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.201811:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0db5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77545472->77549567] (req 77545472->77549567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.203957:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8688us (8891us total) trans 4294986239 rc 0/0 00000100:00100000:2.0:1789760781.203972:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18943 00000100:00100000:2.0:1789760781.203982:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711307019137 time 711319853027 state 0x2 ref 0 00000100:00100000:2.0:1789760781.210979:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.211081:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811712 00000100:00100000:1.0:1789760781.211091:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811712 00000100:00100000:1.0:1789760781.211144:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711307019137 time 711319853027 state 0x0 ref 1 00000100:00100000:1.0:1789760781.211159:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711320113443 time 711319853027 state 0x1 ref 1 00000100:00100000:1.0:1789760781.211190:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711320113443 time 711335796299 state 0x0 ref 1 00000100:00100000:1.0:1789760781.211200:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.211244:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77549568, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fc0 00010000:00010000:1.0:1789760781.211253:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.211269:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.211273:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.211280:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.211293:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.216473:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986240 is committed 00002000:00010000:1.0:1789760781.216814:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77549568 -> 77553664 00002000:00010000:1.0:1789760781.216820:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151488 -> 151496 00010000:00010000:1.0:1789760781.216834:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.216849:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.216857:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.216865:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.216870:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.216879:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.216898:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.216910:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0dbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77549568->77553663] (req 77549568->77553663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.217007:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+7:2393:x1876699503811712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5802us (6031us total) trans 4294986240 rc 0/0 00000100:00100000:1.0:1789760781.217016:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18944 00000100:00100000:1.0:1789760781.217026:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711320113443 time 711335796299 state 0x2 ref 0 00000100:00100000:1.0:1789760781.220085:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.220177:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811840 00000100:00100000:2.0:1789760781.220187:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811840 00000100:00100000:2.0:1789760781.220243:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711320113443 time 711335796299 state 0x0 ref 1 00000100:00100000:2.0:1789760781.220259:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711336056715 time 711335796299 state 0x1 ref 1 00000100:00100000:2.0:1789760781.220294:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711336056715 time 711344900557 state 0x0 ref 1 00000100:00100000:2.0:1789760781.220318:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.220363:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77553664, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fc8 00010000:00010000:2.0:1789760781.220372:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.220425:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.220431:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.220437:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.220458:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.226940:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986241 is committed 00002000:00010000:2.0:1789760781.227054:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77553664 -> 77557760 00002000:00010000:2.0:1789760781.227059:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151496 -> 151504 00010000:00010000:2.0:1789760781.227072:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227084:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227092:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227098:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.227103:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227110:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227129:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.227137:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77553664->77557759] (req 77553664->77557759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.227234:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503811840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6914us (7155us total) trans 4294986241 rc 0/0 00000100:00100000:2.0:1789760781.227247:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18945 00000100:00100000:2.0:1789760781.227256:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711336056715 time 711344900557 state 0x2 ref 0 00000100:00100000:3.0:1789760781.232457:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.232652:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503811968 00000100:00100000:2.0:1789760781.232661:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503811968 00000100:00100000:2.0:1789760781.232711:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711336056715 time 711344900557 state 0x0 ref 1 00000100:00100000:2.0:1789760781.232725:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711345160973 time 711344900557 state 0x1 ref 1 00000100:00100000:2.0:1789760781.232754:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711345160973 time 711357361087 state 0x0 ref 1 00000100:00100000:2.0:1789760781.232764:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503811968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.232808:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77557760, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fd0 00010000:00010000:2.0:1789760781.232817:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.232832:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.232836:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.232842:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.232855:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.237249:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986242 is committed 00002000:00010000:2.0:1789760781.237456:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77557760 -> 77561856 00002000:00010000:2.0:1789760781.237462:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151504 -> 151512 00010000:00010000:2.0:1789760781.237474:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.237487:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.237496:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.237504:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.237509:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.237518:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.237543:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.237550:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77557760->77561855] (req 77557760->77561855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.237753:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503811968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4984us (5300us total) trans 4294986242 rc 0/0 00000100:00100000:2.0:1789760781.237764:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18946 00000100:00100000:2.0:1789760781.237773:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711345160973 time 711357361087 state 0x2 ref 0 00000100:00100000:1.0:1789760781.239863:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.240046:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812096 00000100:00100000:0.0:1789760781.240055:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812096 00000100:00100000:0.0:1789760781.240107:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711345160973 time 711357361087 state 0x0 ref 1 00000100:00100000:0.0:1789760781.240122:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711357621503 time 711357361087 state 0x1 ref 1 00000100:00100000:0.0:1789760781.240152:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711357621503 time 711364759043 state 0x0 ref 1 00000100:00100000:0.0:1789760781.240164:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503812096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.240207:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77561856, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fd8 00010000:00010000:0.0:1789760781.240215:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.240230:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760781.240235:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.240242:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.240255:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.248797:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986243 is committed 00002000:00010000:1.0:1789760781.248995:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77561856 -> 77565952 00002000:00010000:1.0:1789760781.249000:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151512 -> 151520 00010000:00010000:1.0:1789760781.249014:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.249029:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.249039:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.249046:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.249050:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.249065:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.249085:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.249093:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0dd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77561856->77565951] (req 77561856->77565951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.249232:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503812096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9064us (9371us total) trans 4294986243 rc 0/0 00000100:00100000:1.0:1789760781.249242:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18947 00000100:00100000:1.0:1789760781.249253:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711357621503 time 711364759043 state 0x2 ref 0 00000100:00100000:1.0:1789760781.253077:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.253189:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812224 00000100:00100000:2.0:1789760781.253198:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812224 00000100:00100000:2.0:1789760781.253256:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711357621503 time 711364759043 state 0x0 ref 1 00000100:00100000:2.0:1789760781.253280:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711365019459 time 711364759043 state 0x1 ref 1 00000100:00100000:2.0:1789760781.253442:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711365019459 time 711378047566 state 0x0 ref 1 00000100:00100000:2.0:1789760781.253456:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.253505:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77565952, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fe0 00010000:00010000:2.0:1789760781.253514:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.253529:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.253534:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.253541:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.253554:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.258654:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986244 is committed 00002000:00010000:2.0:1789760781.258977:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77565952 -> 77570048 00002000:00010000:2.0:1789760781.258984:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151520 -> 151528 00010000:00010000:2.0:1789760781.258997:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.259011:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.259020:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.259028:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.259032:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.259042:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.259062:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.259069:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0dd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77565952->77570047] (req 77565952->77570047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.259213:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5755us (6140us total) trans 4294986244 rc 0/0 00000100:00100000:2.0:1789760781.259224:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18948 00000100:00100000:2.0:1789760781.259234:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711365019459 time 711378047566 state 0x2 ref 0 00000100:00100000:2.0:1789760781.264107:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.264198:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812352 00000100:00100000:0.0:1789760781.264209:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812352 00000100:00100000:0.0:1789760781.264265:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711365019459 time 711378047566 state 0x0 ref 1 00000100:00100000:0.0:1789760781.264281:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711378307982 time 711378047566 state 0x1 ref 1 00000100:00100000:0.0:1789760781.264316:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711378307982 time 711388922093 state 0x0 ref 1 00000100:00100000:0.0:1789760781.264330:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503812352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.264558:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77570048, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24fe8 00010000:00010000:0.0:1789760781.264643:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.264661:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760781.264666:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.264672:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.264751:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.268943:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986245 is committed 00002000:00010000:2.0:1789760781.269078:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77570048 -> 77574144 00002000:00010000:2.0:1789760781.269082:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151528 -> 151536 00010000:00010000:2.0:1789760781.269093:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.269102:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.269108:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.269113:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.269117:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.269123:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.269138:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.269142:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0ddf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77570048->77574143] (req 77570048->77574143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.269273:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503812352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4939us (5170us total) trans 4294986245 rc 0/0 00000100:00100000:2.0:1789760781.269285:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18949 00000100:00100000:2.0:1789760781.269305:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711378307982 time 711388922093 state 0x2 ref 0 00000100:00100000:1.0:1789760781.276868:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.277013:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812480 00000100:00100000:3.0:1789760781.277021:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812480 00000100:00100000:3.0:1789760781.277080:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711378307982 time 711388922093 state 0x0 ref 1 00000100:00100000:3.0:1789760781.277094:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711389182509 time 711388922093 state 0x1 ref 1 00000100:00100000:2.0:1789760781.277292:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711389182509 time 711401896911 state 0x0 ref 1 00000100:00100000:2.0:1789760781.277308:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.277354:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77574144, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24ff0 00010000:00010000:2.0:1789760781.277362:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.277377:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.277382:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.277417:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.277433:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.283065:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986246 is committed 00002000:00010000:2.0:1789760781.284020:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77574144 -> 77578240 00002000:00010000:2.0:1789760781.284027:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151536 -> 151544 00010000:00010000:2.0:1789760781.284040:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.284052:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.284066:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.284074:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.284078:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.284088:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.284107:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.284115:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f0de6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77574144->77578239] (req 77574144->77578239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.284220:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6910us (7355us total) trans 4294986246 rc 0/0 00000100:00100000:2.0:1789760781.284230:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18950 00000100:00100000:2.0:1789760781.284239:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711389182509 time 711401896911 state 0x2 ref 0 00000100:00100000:1.0:1789760781.287595:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.287693:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812608 00000100:00100000:2.0:1789760781.287702:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812608 00000100:00100000:2.0:1789760781.287757:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711389182509 time 711401896911 state 0x0 ref 1 00000100:00100000:2.0:1789760781.287772:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711402157327 time 711401896911 state 0x1 ref 1 00000100:00100000:2.0:1789760781.287845:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711402157327 time 711412451046 state 0x0 ref 1 00000100:00100000:2.0:1789760781.287858:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503812608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.287917:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77578240, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x24ff8 00010000:00010000:2.0:1789760781.287926:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.287951:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.287956:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.287962:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.287976:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.293135:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986247 is committed 00002000:00010000:2.0:1789760781.293244:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77578240 -> 77582336 00002000:00010000:2.0:1789760781.293249:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151544 -> 151552 00010000:00010000:2.0:1789760781.293262:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.293373:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.293383:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.293471:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.293476:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.293485:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.293505:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.293623:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0ded lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77578240->77582335] (req 77578240->77582335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.293710:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503812608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5850us (6119us total) trans 4294986247 rc 0/0 00000100:00100000:2.0:1789760781.293720:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18951 00000100:00100000:2.0:1789760781.293730:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711402157327 time 711412451046 state 0x2 ref 0 00000100:00100000:1.0:1789760781.296627:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.296788:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812736 00000100:00100000:3.0:1789760781.296798:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812736 00000100:00100000:3.0:1789760781.296852:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711402157327 time 711412451046 state 0x0 ref 1 00000100:00100000:3.0:1789760781.297247:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711412711462 time 711412451046 state 0x1 ref 1 00000100:00100000:3.0:1789760781.297278:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711412711462 time 711421884266 state 0x0 ref 1 00000100:00100000:3.0:1789760781.297292:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.297354:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77582336, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25000 00010000:00010000:3.0:1789760781.297365:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.297380:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.297385:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.297413:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.297427:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.305042:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986248 is committed 00002000:00010000:3.0:1789760781.305164:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77582336 -> 77586432 00002000:00010000:3.0:1789760781.305169:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151552 -> 151560 00010000:00010000:3.0:1789760781.305182:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.305195:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.305204:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.305211:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.305216:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.305231:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.305260:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.305275:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0df4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77582336->77586431] (req 77582336->77586431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.305473:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8178us (8852us total) trans 4294986248 rc 0/0 00000100:00100000:3.0:1789760781.305485:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18952 00000100:00100000:3.0:1789760781.305495:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711412711462 time 711421884266 state 0x2 ref 0 00000100:00100000:1.0:1789760781.309150:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.310481:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812864 00000100:00100000:0.0:1789760781.310497:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812864 00000100:00100000:0.0:1789760781.310551:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711412711462 time 711421884266 state 0x0 ref 1 00000100:00100000:0.0:1789760781.310566:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711422144682 time 711421884266 state 0x1 ref 1 00000100:00100000:0.0:1789760781.310663:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711422144682 time 711435268745 state 0x0 ref 1 00000100:00100000:0.0:1789760781.310678:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.310723:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77586432, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25008 00010000:00010000:0.0:1789760781.310732:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.310746:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760781.310750:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.310756:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.310768:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.316466:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986249 is committed 00002000:00010000:0.0:1789760781.317045:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77586432 -> 77590528 00002000:00010000:0.0:1789760781.317051:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151560 -> 151568 00010000:00010000:0.0:1789760781.317063:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.317075:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.317084:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.317386:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.317414:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.317424:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.317445:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.317453:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0dfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77586432->77590527] (req 77586432->77590527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.317598:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6918us (8449us total) trans 4294986249 rc 0/0 00000100:00100000:0.0:1789760781.317609:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18953 00000100:00100000:0.0:1789760781.317618:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711422144682 time 711435268745 state 0x2 ref 0 00000100:00100000:1.0:1789760781.325660:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.325773:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503812992 00000100:00100000:2.0:1789760781.325782:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503812992 00000100:00100000:2.0:1789760781.325847:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711422144682 time 711435268745 state 0x0 ref 1 00000100:00100000:2.0:1789760781.325863:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711435529161 time 711435268745 state 0x1 ref 1 00000100:00100000:2.0:1789760781.325896:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711435529161 time 711450503046 state 0x0 ref 1 00000100:00100000:2.0:1789760781.325909:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.325954:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77590528, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25010 00010000:00010000:2.0:1789760781.325963:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.325978:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.325983:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.325989:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.326002:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.333537:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986250 is committed 00002000:00010000:1.0:1789760781.333812:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77590528 -> 77594624 00002000:00010000:1.0:1789760781.333818:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151568 -> 151576 00010000:00010000:1.0:1789760781.333831:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.333844:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.333854:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.333862:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.333866:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.333875:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.333894:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.333903:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f0e02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77590528->77594623] (req 77590528->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.333995:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503812992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8084us (8600us total) trans 4294986250 rc 0/0 00000100:00100000:1.0:1789760781.334005:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18954 00000100:00100000:1.0:1789760781.334024:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711435529161 time 711450503046 state 0x2 ref 0 00000100:00100000:1.0:1789760781.337273:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.337365:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503813120 00000100:00100000:2.0:1789760781.337375:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503813120 00000100:00100000:2.0:1789760781.337469:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711435529161 time 711450503046 state 0x0 ref 1 00000100:00100000:2.0:1789760781.337484:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711450763462 time 711450503046 state 0x1 ref 1 00000100:00100000:2.0:1789760781.337505:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711450763462 time 711462111904 state 0x0 ref 1 00000100:00100000:2.0:1789760781.337517:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.337566:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77594624, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25018 00010000:00010000:2.0:1789760781.337587:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.337604:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.337609:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.337615:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.337628:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.345181:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986251 is committed 00002000:00010000:2.0:1789760781.345296:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77594624 -> 77598720 00002000:00010000:2.0:1789760781.345302:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151576 -> 151584 00010000:00010000:2.0:1789760781.345315:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.345327:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.345337:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.345344:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.345348:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.345357:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.345478:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.345488:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77594624->77598719] (req 77594624->77598719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.345654:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8133us (8384us total) trans 4294986251 rc 0/0 00000100:00100000:2.0:1789760781.345665:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18955 00000100:00100000:2.0:1789760781.345675:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711450763462 time 711462111904 state 0x2 ref 0 00000100:00100000:2.0:1789760781.350690:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.350764:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503813248 00000100:00100000:3.0:1789760781.350773:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503813248 00000100:00100000:3.0:1789760781.350827:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711450763462 time 711462111904 state 0x0 ref 1 00000100:00100000:3.0:1789760781.350842:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711462372320 time 711462111904 state 0x1 ref 1 00000100:00100000:3.0:1789760781.350873:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711462372320 time 711475480207 state 0x0 ref 1 00000100:00100000:3.0:1789760781.350885:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503813248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.350929:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77598720, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25020 00010000:00010000:3.0:1789760781.350950:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.350965:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.350969:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.350975:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.350990:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.355859:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986252 is committed 00002000:00010000:3.0:1789760781.356799:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77598720 -> 77602816 00002000:00010000:3.0:1789760781.356807:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151584 -> 151592 00010000:00010000:3.0:1789760781.356820:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.356832:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.356841:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.356849:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.356854:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.356863:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.356883:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.356891:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77598720->77602815] (req 77598720->77602815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.357133:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503813248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6244us (6447us total) trans 4294986252 rc 0/0 00000100:00100000:3.0:1789760781.357143:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18956 00000100:00100000:3.0:1789760781.357152:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711462372320 time 711475480207 state 0x2 ref 0 00000100:00100000:1.0:1789760781.362691:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.362849:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503813376 00000100:00100000:2.0:1789760781.362859:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503813376 00000100:00100000:2.0:1789760781.362913:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711462372320 time 711475480207 state 0x0 ref 1 00000100:00100000:2.0:1789760781.362928:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711475740623 time 711475480207 state 0x1 ref 1 00000100:00100000:2.0:1789760781.362959:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711475740623 time 711487565485 state 0x0 ref 1 00000100:00100000:2.0:1789760781.362973:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503813376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.363031:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77602816, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25028 00010000:00010000:2.0:1789760781.363040:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.363061:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.363066:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.363072:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.363085:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.370628:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986253 is committed 00002000:00010000:2.0:1789760781.370744:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77602816 -> 77606912 00002000:00010000:2.0:1789760781.370749:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151592 -> 151600 00010000:00010000:2.0:1789760781.370763:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.370774:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.370783:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.370790:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.370795:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.370804:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.370828:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.370836:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77602816->77606911] (req 77602816->77606911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.370971:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503813376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7995us (8283us total) trans 4294986253 rc 0/0 00000100:00100000:2.0:1789760781.370982:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18957 00000100:00100000:2.0:1789760781.371004:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711475740623 time 711487565485 state 0x2 ref 0 00000100:00100000:1.0:1789760781.377848:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.378205:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503813504 00000100:00100000:2.0:1789760781.378215:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503813504 00000100:00100000:2.0:1789760781.378269:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711475740623 time 711487565485 state 0x0 ref 1 00000100:00100000:2.0:1789760781.378285:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711487825901 time 711487565485 state 0x1 ref 1 00000100:00100000:2.0:1789760781.378449:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711487825901 time 711503053646 state 0x0 ref 1 00000100:00100000:2.0:1789760781.378463:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.378508:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77606912, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25030 00010000:00010000:2.0:1789760781.378516:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.378531:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.378537:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.378543:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.378557:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.383255:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986254 is committed 00002000:00010000:2.0:1789760781.383555:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77606912 -> 77611008 00002000:00010000:2.0:1789760781.383560:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151600 -> 151608 00010000:00010000:2.0:1789760781.383584:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383597:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383606:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383680:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.383685:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383694:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383714:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383727:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0e1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77606912->77611007] (req 77606912->77611007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.383863:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5397us (6019us total) trans 4294986254 rc 0/0 00000100:00100000:2.0:1789760781.383873:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18958 00000100:00100000:2.0:1789760781.383883:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711487825901 time 711503053646 state 0x2 ref 0 00000100:00100000:1.0:1789760781.386206:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.386365:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503813632 00000100:00100000:2.0:1789760781.386375:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503813632 00000100:00100000:2.0:1789760781.386734:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711487825901 time 711503053646 state 0x0 ref 1 00000100:00100000:2.0:1789760781.386749:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711503314062 time 711503053646 state 0x1 ref 1 00000100:00100000:2.0:1789760781.386819:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711503314062 time 711511425316 state 0x0 ref 1 00000100:00100000:2.0:1789760781.386833:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.386878:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77611008, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25038 00010000:00010000:2.0:1789760781.386887:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.386902:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.386906:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.386912:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.386930:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.391454:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986255 is committed 00002000:00010000:1.0:1789760781.391694:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77611008 -> 77615104 00002000:00010000:1.0:1789760781.391700:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151608 -> 151616 00010000:00010000:1.0:1789760781.391713:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.391725:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.391734:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.391742:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.391746:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.391755:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.391775:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.391783:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0e25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77611008->77615103] (req 77611008->77615103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.392088:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5253us (5886us total) trans 4294986255 rc 0/0 00000100:00100000:1.0:1789760781.392099:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18959 00000100:00100000:1.0:1789760781.392109:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711503314062 time 711511425316 state 0x2 ref 0 00000100:00100000:1.0:1789760781.396258:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.396829:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503813760 00000100:00100000:3.0:1789760781.396839:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503813760 00000100:00100000:3.0:1789760781.396895:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711503314062 time 711511425316 state 0x0 ref 1 00000100:00100000:3.0:1789760781.396912:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711511685732 time 711511425316 state 0x1 ref 1 00000100:00100000:3.0:1789760781.396938:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711511685732 time 711521545154 state 0x0 ref 1 00000100:00100000:3.0:1789760781.396952:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.396994:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77615104, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25040 00010000:00010000:3.0:1789760781.397002:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.397017:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.397022:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.397028:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.397042:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.403268:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986256 is committed 00002000:00010000:3.0:1789760781.407288:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77615104 -> 77619200 00002000:00010000:3.0:1789760781.407360:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151616 -> 151624 00010000:00010000:3.0:1789760781.407375:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.407403:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.407413:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.407422:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.407427:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.407437:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.407458:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.407465:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77615104->77619199] (req 77615104->77619199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.407629:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503813760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10675us (11375us total) trans 4294986256 rc 0/0 00000100:00100000:3.0:1789760781.407640:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18960 00000100:00100000:3.0:1789760781.407649:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711511685732 time 711521545154 state 0x2 ref 0 00000100:00100000:2.0:1789760781.414170:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.414258:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503813888 00000100:00100000:0.0:1789760781.414269:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503813888 00000100:00100000:0.0:1789760781.414329:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711511685732 time 711521545154 state 0x0 ref 1 00000100:00100000:0.0:1789760781.414344:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711521805570 time 711521545154 state 0x1 ref 1 00000100:00100000:0.0:1789760781.414372:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711521805570 time 711538978838 state 0x0 ref 1 00000100:00100000:0.0:1789760781.414384:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503813888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.414458:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77619200, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25048 00010000:00010000:0.0:1789760781.414468:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.414483:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760781.414488:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.414494:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.414508:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.418110:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986257 is committed 00002000:00010000:0.0:1789760781.418223:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77619200 -> 77623296 00002000:00010000:0.0:1789760781.418228:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151624 -> 151632 00010000:00010000:0.0:1789760781.418240:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.418251:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.418260:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.418267:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.418271:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.418281:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.418309:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.418316:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f0e33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77619200->77623295] (req 77619200->77623295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.418590:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503813888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4202us (4424us total) trans 4294986257 rc 0/0 00000100:00100000:0.0:1789760781.418600:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18961 00000100:00100000:0.0:1789760781.418610:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711521805570 time 711538978838 state 0x2 ref 0 00000100:00100000:1.0:1789760781.422005:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.422100:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814016 00000100:00100000:2.0:1789760781.422110:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814016 00000100:00100000:2.0:1789760781.422164:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711521805570 time 711538978838 state 0x0 ref 1 00000100:00100000:2.0:1789760781.422179:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711539239254 time 711538978838 state 0x1 ref 1 00000100:00100000:2.0:1789760781.422210:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711539239254 time 711546816056 state 0x0 ref 1 00000100:00100000:2.0:1789760781.422222:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503814016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.422267:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77623296, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25050 00010000:00010000:2.0:1789760781.422276:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.422291:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.422307:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.422313:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.422327:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.429914:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986258 is committed 00002000:00010000:2.0:1789760781.430033:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77623296 -> 77627392 00002000:00010000:2.0:1789760781.430038:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151632 -> 151640 00010000:00010000:2.0:1789760781.430051:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.430063:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.430071:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.430078:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.430083:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.430092:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.430112:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.430119:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77623296->77627391] (req 77623296->77627391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.430256:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503814016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8031us (8255us total) trans 4294986258 rc 0/0 00000100:00100000:2.0:1789760781.430268:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18962 00000100:00100000:2.0:1789760781.430277:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711539239254 time 711546816056 state 0x2 ref 0 00000100:00100000:1.0:1789760781.438717:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.438800:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814144 00000100:00100000:2.0:1789760781.438809:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814144 00000100:00100000:2.0:1789760781.438861:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711539239254 time 711546816056 state 0x0 ref 1 00000100:00100000:2.0:1789760781.438875:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711547076472 time 711546816056 state 0x1 ref 1 00000100:00100000:2.0:1789760781.438956:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711547076472 time 711563562126 state 0x0 ref 1 00000100:00100000:2.0:1789760781.438966:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503814144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.439003:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77627392, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25058 00010000:00010000:2.0:1789760781.439011:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.439024:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.439027:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.439030:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.439040:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.444061:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986259 is committed 00002000:00010000:1.0:1789760781.444229:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77627392 -> 77631488 00002000:00010000:1.0:1789760781.444235:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151640 -> 151648 00010000:00010000:1.0:1789760781.444251:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.444264:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.444272:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.444279:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.444284:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.444292:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.444312:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.444320:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f0e41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77627392->77631487] (req 77627392->77631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.444449:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503814144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5478us (5734us total) trans 4294986259 rc 0/0 00000100:00100000:1.0:1789760781.444461:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18963 00000100:00100000:1.0:1789760781.444470:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711547076472 time 711563562126 state 0x2 ref 0 00000100:00100000:2.0:1789760781.449370:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.449527:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814272 00000100:00100000:3.0:1789760781.449536:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814272 00000100:00100000:3.0:1789760781.449591:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711547076472 time 711563562126 state 0x0 ref 1 00000100:00100000:3.0:1789760781.449606:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711563822542 time 711563562126 state 0x1 ref 1 00000100:00100000:3.0:1789760781.449727:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711563822542 time 711574331627 state 0x0 ref 1 00000100:00100000:3.0:1789760781.449739:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503814272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.449784:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77631488, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25060 00010000:00010000:3.0:1789760781.449792:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.449808:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.449813:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.449819:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.449832:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.453564:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986260 is committed 00002000:00010000:3.0:1789760781.453710:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77631488 -> 77635584 00002000:00010000:3.0:1789760781.453716:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151648 -> 151656 00010000:00010000:3.0:1789760781.453729:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.453741:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.453750:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.453757:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.453762:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.453772:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.453792:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.453799:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77631488->77635583] (req 77631488->77635583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.453897:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503814272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4155us (4532us total) trans 4294986260 rc 0/0 00000100:00100000:3.0:1789760781.453905:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18964 00000100:00100000:3.0:1789760781.453916:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711563822542 time 711574331627 state 0x2 ref 0 00000100:00100000:2.0:1789760781.459861:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.459987:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814400 00000100:00100000:3.0:1789760781.459998:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814400 00000100:00100000:3.0:1789760781.460051:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711563822542 time 711574331627 state 0x0 ref 1 00000100:00100000:3.0:1789760781.460067:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711574592043 time 711574331627 state 0x1 ref 1 00000100:00100000:3.0:1789760781.460105:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711574592043 time 711584711020 state 0x0 ref 1 00000100:00100000:3.0:1789760781.460120:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503814400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.460184:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77635584, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25068 00010000:00010000:3.0:1789760781.460196:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.460213:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.460218:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.460224:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.460240:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.465521:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986261 is committed 00002000:00010000:3.0:1789760781.465662:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77635584 -> 77639680 00002000:00010000:3.0:1789760781.465668:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151656 -> 151664 00010000:00010000:3.0:1789760781.465680:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.465697:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.465705:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.465713:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.465717:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.465727:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.465750:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.465756:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77635584->77639679] (req 77635584->77639679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.466009:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503814400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5877us (6144us total) trans 4294986261 rc 0/0 00000100:00100000:3.0:1789760781.466020:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18965 00000100:00100000:3.0:1789760781.466031:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711574592043 time 711584711020 state 0x2 ref 0 00000100:00100000:1.0:1789760781.473341:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.474697:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814528 00000100:00100000:2.0:1789760781.474711:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814528 00000100:00100000:2.0:1789760781.474778:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711574592043 time 711584711020 state 0x0 ref 1 00000100:00100000:2.0:1789760781.474793:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711584971436 time 711584711020 state 0x1 ref 1 00000100:00100000:2.0:1789760781.474829:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711584971436 time 711599435499 state 0x0 ref 1 00000100:00100000:2.0:1789760781.474841:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503814528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.474890:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77639680, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25070 00010000:00010000:2.0:1789760781.474899:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.474925:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.474931:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.474939:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.474954:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.480811:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986262 is committed 00002000:00010000:2.0:1789760781.483268:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77639680 -> 77643776 00002000:00010000:2.0:1789760781.483277:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151664 -> 151672 00010000:00010000:2.0:1789760781.483299:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.483317:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.483326:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.483333:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.483337:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.483346:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.483363:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.483370:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f0e56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77639680->77643775] (req 77639680->77643775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.483511:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503814528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8667us (10175us total) trans 4294986262 rc 0/0 00000100:00100000:2.0:1789760781.483521:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18966 00000100:00100000:2.0:1789760781.483530:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711584971436 time 711599435499 state 0x2 ref 0 00000100:00100000:3.0:1789760781.488739:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.488833:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814656 00000100:00100000:1.0:1789760781.488844:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814656 00000100:00100000:1.0:1789760781.488899:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711584971436 time 711599435499 state 0x0 ref 1 00000100:00100000:1.0:1789760781.488967:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711599695915 time 711599435499 state 0x1 ref 1 00000100:00100000:1.0:1789760781.489036:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711599695915 time 711613641725 state 0x0 ref 1 00000100:00100000:1.0:1789760781.489050:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503814656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.489096:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77643776, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25078 00010000:00010000:1.0:1789760781.489106:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.489132:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.489136:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.489143:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.489161:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.495744:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986263 is committed 00002000:00010000:1.0:1789760781.495859:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77643776 -> 77647872 00002000:00010000:1.0:1789760781.495865:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151672 -> 151680 00010000:00010000:1.0:1789760781.495878:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.495890:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.495899:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.495906:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.495911:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.495929:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.495947:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.495955:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f0e5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77643776->77647871] (req 77643776->77647871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.496048:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503814656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6995us (7312us total) trans 4294986263 rc 0/0 00000100:00100000:1.0:1789760781.496059:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18967 00000100:00100000:1.0:1789760781.496068:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711599695915 time 711613641725 state 0x2 ref 0 00000100:00100000:1.0:1789760781.499819:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.499945:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814784 00000100:00100000:3.0:1789760781.499955:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814784 00000100:00100000:3.0:1789760781.500021:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711599695915 time 711613641725 state 0x0 ref 1 00000100:00100000:3.0:1789760781.500100:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711613902141 time 711613641725 state 0x1 ref 1 00000100:00100000:3.0:1789760781.500131:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711613902141 time 711624738432 state 0x0 ref 1 00000100:00100000:3.0:1789760781.500143:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503814784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.500189:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77647872, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25080 00010000:00010000:3.0:1789760781.500198:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.500213:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.500218:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.500224:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.500237:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.506274:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986264 is committed 00002000:00010000:3.0:1789760781.506451:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77647872 -> 77651968 00002000:00010000:3.0:1789760781.506456:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151680 -> 151688 00010000:00010000:3.0:1789760781.506470:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.506488:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.506497:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.506504:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.506509:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.506517:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.506536:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.506544:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77647872->77651967] (req 77647872->77651967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.507581:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503814784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7424us (7756us total) trans 4294986264 rc 0/0 00000100:00100000:3.0:1789760781.507604:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18968 00000100:00100000:3.0:1789760781.507620:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711613902141 time 711624738432 state 0x2 ref 0 00000100:00100000:2.0:1789760781.511775:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.512021:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503814912 00000100:00100000:3.0:1789760781.512032:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503814912 00000100:00100000:3.0:1789760781.512088:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711613902141 time 711624738432 state 0x0 ref 1 00000100:00100000:3.0:1789760781.512102:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711624998848 time 711624738432 state 0x1 ref 1 00000100:00100000:3.0:1789760781.512207:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711624998848 time 711636813217 state 0x0 ref 1 00000100:00100000:3.0:1789760781.512222:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503814912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.512269:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77651968, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25088 00010000:00010000:3.0:1789760781.512278:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.512293:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.512307:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.512314:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.512329:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.516758:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986265 is committed 00002000:00010000:3.0:1789760781.517089:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77651968 -> 77656064 00002000:00010000:3.0:1789760781.517095:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151688 -> 151696 00010000:00010000:3.0:1789760781.517107:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.517121:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.517137:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.517145:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.517150:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.517160:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.517178:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.517186:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77651968->77656063] (req 77651968->77656063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.517352:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503814912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5128us (5583us total) trans 4294986265 rc 0/0 00000100:00100000:3.0:1789760781.517362:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18969 00000100:00100000:3.0:1789760781.517372:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711624998848 time 711636813217 state 0x2 ref 0 00000100:00100000:1.0:1789760781.521849:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.522079:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815040 00000100:00100000:2.0:1789760781.522089:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815040 00000100:00100000:2.0:1789760781.522140:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711624998848 time 711636813217 state 0x0 ref 1 00000100:00100000:2.0:1789760781.522154:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711637073633 time 711636813217 state 0x1 ref 1 00000100:00100000:2.0:1789760781.522182:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711637073633 time 711646788948 state 0x0 ref 1 00000100:00100000:2.0:1789760781.522193:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.522237:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77656064, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25090 00010000:00010000:2.0:1789760781.522246:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.522261:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.522266:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.522272:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.522285:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.522644:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ccea0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523610496:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760781.522694:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760781.522872:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523610496 00000100:00100000:3.0:1789760781.522878:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523610496 00000100:00100000:3.0:1789760781.522923:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1789760781.522932:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523610496:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760781.522957:0:7941:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:3.0:1789760781.523428:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523610496:12345-0@lo:41:osp-pre-1-0.0 Request processed in 496us (732us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760781.523436:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789760781.523692:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccea0d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523610496:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760781.532842:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986266 is committed 00002000:00010000:2.0:1789760781.533191:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77656064 -> 77660160 00002000:00010000:2.0:1789760781.533197:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151696 -> 151704 00010000:00010000:2.0:1789760781.533211:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.533223:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.533234:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.533242:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.533246:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.533255:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.533274:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.533282:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77656064->77660159] (req 77656064->77660159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.533645:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11439us (11791us total) trans 4294986266 rc 0/0 00000100:00100000:2.0:1789760781.533657:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18970 00000100:00100000:2.0:1789760781.533668:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711637073633 time 711646788948 state 0x2 ref 0 00000100:00100000:1.0:1789760781.539016:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.539175:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815168 00000100:00100000:2.0:1789760781.539186:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815168 00000100:00100000:2.0:1789760781.539239:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711637073633 time 711646788948 state 0x0 ref 1 00000100:00100000:2.0:1789760781.539253:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711647049364 time 711646788948 state 0x1 ref 1 00000100:00100000:2.0:1789760781.539282:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711647049364 time 711663889315 state 0x0 ref 1 00000100:00100000:2.0:1789760781.539302:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503815168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.539352:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77660160, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25098 00010000:00010000:2.0:1789760781.539361:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.539376:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.539380:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.539386:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.539429:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.546761:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986267 is committed 00002000:00010000:2.0:1789760781.546917:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77660160 -> 77664256 00002000:00010000:2.0:1789760781.546923:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151704 -> 151712 00010000:00010000:2.0:1789760781.546935:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.546949:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.546957:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.546979:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.546985:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.546994:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.547013:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.547021:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77660160->77664255] (req 77660160->77664255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.547117:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503815168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7820us (8106us total) trans 4294986267 rc 0/0 00000100:00100000:2.0:1789760781.547127:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18971 00000100:00100000:2.0:1789760781.547136:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711647049364 time 711663889315 state 0x2 ref 0 00000100:00100000:1.0:1789760781.552134:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.552226:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815296 00000100:00100000:3.0:1789760781.552238:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815296 00000100:00100000:3.0:1789760781.552299:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711647049364 time 711663889315 state 0x0 ref 1 00000100:00100000:3.0:1789760781.552314:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711664149731 time 711663889315 state 0x1 ref 1 00000100:00100000:3.0:1789760781.552342:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711664149731 time 711676949061 state 0x0 ref 1 00000100:00100000:3.0:1789760781.552353:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.552433:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77664256, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250a0 00010000:00010000:3.0:1789760781.552442:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.552457:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.552461:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.552468:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.552483:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.561171:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986268 is committed 00002000:00010000:2.0:1789760781.564490:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77664256 -> 77668352 00002000:00010000:2.0:1789760781.564500:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151712 -> 151720 00010000:00010000:2.0:1789760781.564514:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.564525:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.564539:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.564545:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.564550:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.564558:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.564587:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.564594:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f0e80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77664256->77668351] (req 77664256->77668351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.564695:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12339us (12567us total) trans 4294986268 rc 0/0 00000100:00100000:2.0:1789760781.564706:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18972 00000100:00100000:2.0:1789760781.564715:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711664149731 time 711676949061 state 0x2 ref 0 00000100:00100000:1.0:1789760781.571212:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.571303:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815424 00000100:00100000:3.0:1789760781.571315:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815424 00000100:00100000:3.0:1789760781.571367:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711664149731 time 711676949061 state 0x0 ref 1 00000100:00100000:3.0:1789760781.571381:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711677209477 time 711676949061 state 0x1 ref 1 00000100:00100000:3.0:1789760781.571440:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711677209477 time 711696046976 state 0x0 ref 1 00000100:00100000:3.0:1789760781.571453:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.571502:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77668352, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250a8 00010000:00010000:3.0:1789760781.571512:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.571529:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.571533:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.571540:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.571554:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.581433:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986269 is committed 00002000:00010000:3.0:1789760781.581544:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77668352 -> 77672448 00002000:00010000:3.0:1789760781.581549:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151720 -> 151728 00010000:00010000:3.0:1789760781.581561:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.581592:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.581601:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.581617:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.581622:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.581631:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.581651:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.581659:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0e87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77668352->77672447] (req 77668352->77672447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.581753:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10297us (10548us total) trans 4294986269 rc 0/0 00000100:00100000:3.0:1789760781.581763:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18973 00000100:00100000:3.0:1789760781.581772:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711677209477 time 711696046976 state 0x2 ref 0 00000100:00100000:1.0:1789760781.585518:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.585688:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815552 00000100:00100000:2.0:1789760781.585698:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815552 00000100:00100000:2.0:1789760781.585750:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711677209477 time 711696046976 state 0x0 ref 1 00000100:00100000:2.0:1789760781.585765:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711696307392 time 711696046976 state 0x1 ref 1 00000100:00100000:2.0:1789760781.585838:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711696307392 time 711710443567 state 0x0 ref 1 00000100:00100000:2.0:1789760781.585851:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.585897:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77672448, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250b0 00010000:00010000:2.0:1789760781.585906:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.585922:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.585927:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.585933:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.585947:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.596558:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986270 is committed 00002000:00010000:2.0:1789760781.596777:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77672448 -> 77676544 00002000:00010000:2.0:1789760781.596783:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151728 -> 151736 00010000:00010000:2.0:1789760781.596798:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.596810:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.596819:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.596834:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.596838:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.596848:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.596868:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.597108:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0e8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77672448->77676543] (req 77672448->77676543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.597236:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11382us (11841us total) trans 4294986270 rc 0/0 00000100:00100000:2.0:1789760781.597246:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18974 00000100:00100000:2.0:1789760781.597256:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711696307392 time 711710443567 state 0x2 ref 0 00000100:00100000:1.0:1789760781.599790:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.599917:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815680 00000100:00100000:2.0:1789760781.599926:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815680 00000100:00100000:2.0:1789760781.599977:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711696307392 time 711710443567 state 0x0 ref 1 00000100:00100000:2.0:1789760781.599992:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711710703983 time 711710443567 state 0x1 ref 1 00000100:00100000:2.0:1789760781.600021:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711710703983 time 711724627803 state 0x0 ref 1 00000100:00100000:2.0:1789760781.600037:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503815680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.600082:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77676544, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250b8 00010000:00010000:2.0:1789760781.600092:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.600106:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.600110:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.600115:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.600127:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.611269:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986271 is committed 00002000:00010000:2.0:1789760781.611407:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77676544 -> 77680640 00002000:00010000:2.0:1789760781.611413:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151736 -> 151744 00010000:00010000:2.0:1789760781.611426:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.611437:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.611444:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.611451:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.611457:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.611465:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.611484:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.611490:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0e95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77676544->77680639] (req 77676544->77680639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.611588:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503815680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11548us (11803us total) trans 4294986271 rc 0/0 00000100:00100000:2.0:1789760781.611598:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18975 00000100:00100000:2.0:1789760781.611617:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711710703983 time 711724627803 state 0x2 ref 0 00000100:00100000:1.0:1789760781.614724:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.614822:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815808 00000100:00100000:0.0:1789760781.614834:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815808 00000100:00100000:0.0:1789760781.614890:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711710703983 time 711724627803 state 0x0 ref 1 00000100:00100000:0.0:1789760781.614905:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711724888219 time 711724627803 state 0x1 ref 1 00000100:00100000:0.0:1789760781.614933:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711724888219 time 711739536726 state 0x0 ref 1 00000100:00100000:0.0:1789760781.614946:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.614992:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77680640, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250c0 00010000:00010000:0.0:1789760781.615002:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.615017:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760781.615021:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.615027:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.615040:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.622205:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986272 is committed 00002000:00010000:2.0:1789760781.622383:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77680640 -> 77684736 00002000:00010000:2.0:1789760781.622409:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151744 -> 151752 00010000:00010000:2.0:1789760781.622423:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622436:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622446:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622459:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.622464:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622472:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622491:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.622499:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f0e9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77680640->77684735] (req 77680640->77684735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.622748:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503815808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7796us (8026us total) trans 4294986272 rc 0/0 00000100:00100000:2.0:1789760781.622760:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18976 00000100:00100000:2.0:1789760781.622769:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711724888219 time 711739536726 state 0x2 ref 0 00000100:00100000:1.0:1789760781.626100:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.626815:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503815936 00000100:00100000:3.0:1789760781.626826:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503815936 00000100:00100000:3.0:1789760781.626881:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711724888219 time 711739536726 state 0x0 ref 1 00000100:00100000:3.0:1789760781.626896:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711739797142 time 711739536726 state 0x1 ref 1 00000100:00100000:3.0:1789760781.626956:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711739797142 time 711751561705 state 0x0 ref 1 00000100:00100000:3.0:1789760781.626967:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503815936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.627014:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77684736, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250c8 00010000:00010000:3.0:1789760781.627023:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.627037:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.627041:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.627048:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.627062:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.637856:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986273 is committed 00002000:00010000:3.0:1789760781.637957:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77684736 -> 77688832 00002000:00010000:3.0:1789760781.637964:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151752 -> 151760 00010000:00010000:3.0:1789760781.637977:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.637990:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.638006:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.638014:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.638019:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.638029:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.638048:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.638055:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ea3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77684736->77688831] (req 77684736->77688831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.638151:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503815936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11181us (12056us total) trans 4294986273 rc 0/0 00000100:00100000:3.0:1789760781.638162:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18977 00000100:00100000:3.0:1789760781.638235:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711739797142 time 711751561705 state 0x2 ref 0 00000100:00100000:1.0:1789760781.643226:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.644770:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816064 00000100:00100000:2.0:1789760781.644785:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816064 00000100:00100000:2.0:1789760781.644839:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711739797142 time 711751561705 state 0x0 ref 1 00000100:00100000:2.0:1789760781.644853:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711751822121 time 711751561705 state 0x1 ref 1 00000100:00100000:2.0:1789760781.644896:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711751822121 time 711769502770 state 0x0 ref 1 00000100:00100000:2.0:1789760781.644911:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503816064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.644957:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77688832, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250d0 00010000:00010000:2.0:1789760781.644966:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.644981:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.644985:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.644991:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.645005:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.650946:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986274 is committed 00002000:00010000:2.0:1789760781.651072:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77688832 -> 77692928 00002000:00010000:2.0:1789760781.651077:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151760 -> 151768 00010000:00010000:2.0:1789760781.651091:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651103:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651113:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651121:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.651126:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651135:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651215:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651225:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77688832->77692927] (req 77688832->77692927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.651365:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503816064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6451us (8144us total) trans 4294986274 rc 0/0 00000100:00100000:2.0:1789760781.651376:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18978 00000100:00100000:2.0:1789760781.651385:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711751822121 time 711769502770 state 0x2 ref 0 00000100:00100000:1.0:1789760781.654386:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.654909:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816192 00000100:00100000:2.0:1789760781.654919:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816192 00000100:00100000:2.0:1789760781.654977:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711751822121 time 711769502770 state 0x0 ref 1 00000100:00100000:2.0:1789760781.654993:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711769763186 time 711769502770 state 0x1 ref 1 00000100:00100000:2.0:1789760781.655065:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711769763186 time 711779671064 state 0x0 ref 1 00000100:00100000:2.0:1789760781.655079:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.655125:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77692928, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250d8 00010000:00010000:2.0:1789760781.655135:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.655151:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.655157:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.655164:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.655178:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.673765:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986275 is committed 00002000:00010000:2.0:1789760781.674173:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77692928 -> 77697024 00002000:00010000:2.0:1789760781.674179:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151768 -> 151776 00010000:00010000:2.0:1789760781.674191:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.674203:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.674213:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.674220:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.674225:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.674233:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.674252:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.674260:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0eb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77692928->77697023] (req 77692928->77697023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.674350:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19270us (19969us total) trans 4294986275 rc 0/0 00000100:00100000:2.0:1789760781.674360:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18979 00000100:00100000:2.0:1789760781.674369:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711769763186 time 711779671064 state 0x2 ref 0 00000100:00100000:0.0:1789760781.678271:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.678460:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816320 00000100:00100000:2.0:1789760781.678471:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816320 00000100:00100000:2.0:1789760781.678522:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711769763186 time 711779671064 state 0x0 ref 1 00000100:00100000:2.0:1789760781.678537:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711779931480 time 711779671064 state 0x1 ref 1 00000100:00100000:2.0:1789760781.678566:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711779931480 time 711803172816 state 0x0 ref 1 00000100:00100000:2.0:1789760781.678587:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503816320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.678634:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77697024, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250e0 00010000:00010000:2.0:1789760781.678644:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.678659:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.678665:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.678671:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.678683:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.688921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986276 is committed 00002000:00010000:2.0:1789760781.689037:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77697024 -> 77701120 00002000:00010000:2.0:1789760781.689043:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151776 -> 151784 00010000:00010000:2.0:1789760781.689056:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.689069:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.689078:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.689086:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.689090:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.689099:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.689119:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.689132:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0eb8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77697024->77701119] (req 77697024->77701119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.689223:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503816320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10633us (10958us total) trans 4294986276 rc 0/0 00000100:00100000:2.0:1789760781.689730:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18980 00000100:00100000:2.0:1789760781.689740:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711779931480 time 711803172816 state 0x2 ref 0 00000100:00100000:0.0:1789760781.697050:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.698932:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816448 00000100:00100000:3.0:1789760781.698946:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816448 00000100:00100000:3.0:1789760781.699007:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711779931480 time 711803172816 state 0x0 ref 1 00000100:00100000:3.0:1789760781.699022:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711803433232 time 711803172816 state 0x1 ref 1 00000100:00100000:3.0:1789760781.699109:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711803433232 time 711823714350 state 0x0 ref 1 00000100:00100000:3.0:1789760781.699124:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.699170:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77701120, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250e8 00010000:00010000:3.0:1789760781.699255:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.699271:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.699276:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.699282:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.699297:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.707437:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986277 is committed 00002000:00010000:3.0:1789760781.707588:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77701120 -> 77705216 00002000:00010000:3.0:1789760781.707594:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151784 -> 151792 00010000:00010000:3.0:1789760781.707606:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.707626:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.707636:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.707649:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.707654:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.707663:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.707692:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.707700:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0ebf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77701120->77705215] (req 77701120->77705215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.707806:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8677us (10757us total) trans 4294986277 rc 0/0 00000100:00100000:3.0:1789760781.707817:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18981 00000100:00100000:3.0:1789760781.707827:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711803433232 time 711823714350 state 0x2 ref 0 00000100:00100000:1.0:1789760781.711721:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.711827:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816576 00000100:00100000:2.0:1789760781.711836:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816576 00000100:00100000:2.0:1789760781.711892:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711803433232 time 711823714350 state 0x0 ref 1 00000100:00100000:2.0:1789760781.711907:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711823974766 time 711823714350 state 0x1 ref 1 00000100:00100000:2.0:1789760781.711939:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711823974766 time 711836545582 state 0x0 ref 1 00000100:00100000:2.0:1789760781.711952:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.711996:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77705216, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250f0 00010000:00010000:2.0:1789760781.712006:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.712022:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.712026:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.712033:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.712052:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.718719:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986278 is committed 00002000:00010000:2.0:1789760781.720779:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77705216 -> 77709312 00002000:00010000:2.0:1789760781.720789:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151792 -> 151800 00010000:00010000:2.0:1789760781.720803:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.720817:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.720827:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.720834:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.720839:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.720848:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.720870:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.720878:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ec6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77705216->77709311] (req 77705216->77709311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.721015:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9060us (9298us total) trans 4294986278 rc 0/0 00000100:00100000:2.0:1789760781.721026:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18982 00000100:00100000:2.0:1789760781.721035:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711823974766 time 711836545582 state 0x2 ref 0 00000100:00100000:0.0:1789760781.730799:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.730900:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816704 00000100:00100000:2.0:1789760781.730911:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816704 00000100:00100000:2.0:1789760781.730968:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711823974766 time 711836545582 state 0x0 ref 1 00000100:00100000:2.0:1789760781.730984:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711836805998 time 711836545582 state 0x1 ref 1 00000100:00100000:2.0:1789760781.731008:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711836805998 time 711855614760 state 0x0 ref 1 00000100:00100000:2.0:1789760781.731025:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503816704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.731072:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77709312, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x250f8 00010000:00010000:2.0:1789760781.731081:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.731098:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.731103:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.731109:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.731123:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.741000:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986279 is committed 00002000:00010000:2.0:1789760781.741511:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77709312 -> 77713408 00002000:00010000:2.0:1789760781.741518:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151800 -> 151808 00010000:00010000:2.0:1789760781.741532:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.741545:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.741553:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.741561:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.741565:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.741646:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.741666:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.741675:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ecd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77709312->77713407] (req 77709312->77713407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.741864:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503816704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10836us (11069us total) trans 4294986279 rc 0/0 00000100:00100000:3.0:1789760781.741876:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18983 00000100:00100000:3.0:1789760781.741886:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711836805998 time 711855614760 state 0x2 ref 0 00000100:00100000:0.0:1789760781.747005:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.747855:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816832 00000100:00100000:2.0:1789760781.747868:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816832 00000100:00100000:2.0:1789760781.747928:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711836805998 time 711855614760 state 0x0 ref 1 00000100:00100000:2.0:1789760781.747944:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711855875176 time 711855614760 state 0x1 ref 1 00000100:00100000:2.0:1789760781.747979:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711855875176 time 711872586090 state 0x0 ref 1 00000100:00100000:2.0:1789760781.747993:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.748049:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77713408, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25100 00010000:00010000:2.0:1789760781.748055:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.748066:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.748070:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.748077:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.748087:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.758345:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986280 is committed 00002000:00010000:2.0:1789760781.760794:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77713408 -> 77717504 00002000:00010000:2.0:1789760781.760808:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151808 -> 151816 00010000:00010000:2.0:1789760781.760825:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760838:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760851:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760858:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.760863:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760872:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760891:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760899:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0ed4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77713408->77717503] (req 77713408->77717503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.761027:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13032us (14027us total) trans 4294986280 rc 0/0 00000100:00100000:2.0:1789760781.761038:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18984 00000100:00100000:2.0:1789760781.761048:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711855875176 time 711872586090 state 0x2 ref 0 00000100:00100000:1.0:1789760781.764735:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.764791:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503816960 00000100:00100000:3.0:1789760781.764802:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503816960 00000100:00100000:3.0:1789760781.764860:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711855875176 time 711872586090 state 0x0 ref 1 00000100:00100000:3.0:1789760781.764876:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711872846506 time 711872586090 state 0x1 ref 1 00000100:00100000:3.0:1789760781.764953:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711872846506 time 711889557912 state 0x0 ref 1 00000100:00100000:3.0:1789760781.764966:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.765013:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77717504, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25108 00010000:00010000:3.0:1789760781.765023:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.765039:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.765043:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.765050:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.765063:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.778447:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986281 is committed 00002000:00010000:0.0:1789760781.778547:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77717504 -> 77721600 00002000:00010000:0.0:1789760781.778724:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151816 -> 151824 00010000:00010000:0.0:1789760781.778738:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.778750:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.778759:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.778766:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.778770:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.778779:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.778800:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.778810:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f0edb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77717504->77721599] (req 77717504->77721599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.778957:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503816960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13988us (14225us total) trans 4294986281 rc 0/0 00000100:00100000:0.0:1789760781.778967:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18985 00000100:00100000:0.0:1789760781.778976:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711872846506 time 711889557912 state 0x2 ref 0 00000100:00100000:0.0:1789760781.782015:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.782790:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817088 00000100:00100000:1.0:1789760781.782800:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817088 00000100:00100000:1.0:1789760781.782856:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711872846506 time 711889557912 state 0x0 ref 1 00000100:00100000:1.0:1789760781.782870:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711889818328 time 711889557912 state 0x1 ref 1 00000100:00100000:1.0:1789760781.782902:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711889818328 time 711907509344 state 0x0 ref 1 00000100:00100000:1.0:1789760781.782914:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.783137:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77721600, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x25110 00010000:00010000:1.0:1789760781.783214:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.783230:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.783235:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.783241:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.783255:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.790041:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986282 is committed 00002000:00010000:2.0:1789760781.790208:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77721600 -> 77725696 00002000:00010000:2.0:1789760781.790213:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:2.0:1789760781.790216:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00002000:00010000:2.0:1789760781.790218:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151824 -> 151832 00010000:00010000:2.0:1789760781.790233:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.790246:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.790255:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.790262:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.790267:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.790277:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.790428:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.790438:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f0ee2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77721600->77725695] (req 77721600->77725695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.790603:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7685us (8592us total) trans 4294986282 rc 0/0 00000100:00100000:2.0:1789760781.790614:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18986 00000100:00100000:2.0:1789760781.790623:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711889818328 time 711907509344 state 0x2 ref 0 00000100:00100000:2.0:1789760781.798526:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.798725:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817216 00000100:00100000:1.0:1789760781.798736:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817216 00000100:00100000:1.0:1789760781.798801:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711889818328 time 711907509344 state 0x0 ref 1 00000100:00100000:1.0:1789760781.798816:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711907769760 time 711907509344 state 0x1 ref 1 00000100:00100000:1.0:1789760781.798946:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711907769760 time 711923551431 state 0x0 ref 1 00000100:00100000:1.0:1789760781.798959:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.799020:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77725696, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25118 00010000:00010000:1.0:1789760781.799030:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.799046:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.799050:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.799056:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.799068:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.805267:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986283 is committed 00002000:00010000:3.0:1789760781.805482:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77725696 -> 77729792 00002000:00010000:3.0:1789760781.805488:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151832 -> 151840 00010000:00010000:3.0:1789760781.805501:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805513:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805527:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805533:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.805537:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805545:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805565:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805584:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f0ee9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77725696->77729791] (req 77725696->77729791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.805785:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6821us (7261us total) trans 4294986283 rc 0/0 00000100:00100000:3.0:1789760781.805796:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18987 00000100:00100000:3.0:1789760781.805806:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711907769760 time 711923551431 state 0x2 ref 0 00000100:00100000:1.0:1789760781.809862:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.809953:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817344 00000100:00100000:2.0:1789760781.809964:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817344 00000100:00100000:2.0:1789760781.810020:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711907769760 time 711923551431 state 0x0 ref 1 00000100:00100000:2.0:1789760781.810035:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711923811847 time 711923551431 state 0x1 ref 1 00000100:00100000:2.0:1789760781.810067:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711923811847 time 711934674050 state 0x0 ref 1 00000100:00100000:2.0:1789760781.810079:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.810122:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77729792, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25120 00010000:00010000:2.0:1789760781.810131:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.810146:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.810151:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.810157:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.810170:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.815346:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986284 is committed 00002000:00010000:2.0:1789760781.815591:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77729792 -> 77733888 00002000:00010000:2.0:1789760781.815597:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151840 -> 151848 00010000:00010000:2.0:1789760781.815680:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.815700:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.815709:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.815716:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.815721:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.815730:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.815751:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.815758:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f0ef0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77729792->77733887] (req 77729792->77733887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.815897:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5814us (6050us total) trans 4294986284 rc 0/0 00000100:00100000:2.0:1789760781.815908:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18988 00000100:00100000:2.0:1789760781.815918:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711923811847 time 711934674050 state 0x2 ref 0 00000100:00100000:0.0:1789760781.821564:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.821756:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817472 00000100:00100000:3.0:1789760781.821767:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817472 00000100:00100000:3.0:1789760781.821824:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711923811847 time 711934674050 state 0x0 ref 1 00000100:00100000:3.0:1789760781.821839:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711934934466 time 711934674050 state 0x1 ref 1 00000100:00100000:3.0:1789760781.821905:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711934934466 time 711946510892 state 0x0 ref 1 00000100:00100000:3.0:1789760781.821919:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.822016:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77733888, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25128 00010000:00010000:3.0:1789760781.822025:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.822039:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.822044:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.822050:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.822064:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.828085:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986285 is committed 00002000:00010000:3.0:1789760781.828520:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77733888 -> 77737984 00002000:00010000:3.0:1789760781.828526:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151848 -> 151856 00010000:00010000:3.0:1789760781.828540:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.828552:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.828562:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.828696:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.828701:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.828709:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.828727:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.828735:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0ef7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77733888->77737983] (req 77733888->77737983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.828840:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6919us (7282us total) trans 4294986285 rc 0/0 00000100:00100000:3.0:1789760781.828852:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18989 00000100:00100000:3.0:1789760781.828861:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711934934466 time 711946510892 state 0x2 ref 0 00000100:00100000:2.0:1789760781.835370:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.836498:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817600 00000100:00100000:1.0:1789760781.836512:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817600 00000100:00100000:1.0:1789760781.836593:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711934934466 time 711946510892 state 0x0 ref 1 00000100:00100000:1.0:1789760781.836608:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711946771308 time 711946510892 state 0x1 ref 1 00000100:00100000:1.0:1789760781.836637:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711946771308 time 711961244649 state 0x0 ref 1 00000100:00100000:1.0:1789760781.836650:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc14850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.836695:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77737984, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25130 00010000:00010000:1.0:1789760781.836704:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.836720:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760781.836725:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.836731:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.836745:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.842563:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760781.843250:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986286 is committed 00002000:00010000:1.0:1789760781.843412:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77737984 -> 77742080 00002000:00010000:1.0:1789760781.843417:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151856 -> 151864 00010000:00010000:1.0:1789760781.843430:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.843448:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.843457:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.843464:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.843468:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.843477:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.843497:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.843504:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f0efe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77737984->77742079] (req 77737984->77742079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.843600:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc14850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6947us (8235us total) trans 4294986286 rc 0/0 00000100:00100000:1.0:1789760781.844537:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18990 00000100:00100000:1.0:1789760781.844548:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711946771308 time 711961244649 state 0x2 ref 0 00000100:00100000:1.0:1789760781.847043:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.847126:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817728 00000100:00100000:3.0:1789760781.847136:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817728 00000100:00100000:3.0:1789760781.847192:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711946771308 time 711961244649 state 0x0 ref 1 00000100:00100000:3.0:1789760781.847207:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711961505065 time 711961244649 state 0x1 ref 1 00000100:00100000:3.0:1789760781.847239:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711961505065 time 711971846505 state 0x0 ref 1 00000100:00100000:3.0:1789760781.847254:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.847310:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77742080, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25138 00010000:00010000:3.0:1789760781.847320:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.847335:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760781.847340:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.847347:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.847359:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.851756:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986287 is committed 00002000:00010000:3.0:1789760781.851897:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77742080 -> 77746176 00002000:00010000:3.0:1789760781.851903:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151864 -> 151872 00010000:00010000:3.0:1789760781.851917:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.851931:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.851939:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.851946:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.851951:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.851960:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.851982:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.851989:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77742080->77746175] (req 77742080->77746175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.852147:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4889us (5108us total) trans 4294986287 rc 0/0 00000100:00100000:3.0:1789760781.852157:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18991 00000100:00100000:3.0:1789760781.852168:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711961505065 time 711971846505 state 0x2 ref 0 00000100:00100000:0.0:1789760781.857745:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.857880:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817856 00000100:00100000:2.0:1789760781.857891:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817856 00000100:00100000:2.0:1789760781.857945:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711961505065 time 711971846505 state 0x0 ref 1 00000100:00100000:2.0:1789760781.857960:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711972106921 time 711971846505 state 0x1 ref 1 00000100:00100000:2.0:1789760781.858034:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711972106921 time 711982639732 state 0x0 ref 1 00000100:00100000:2.0:1789760781.858048:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.858097:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77746176, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25140 00010000:00010000:2.0:1789760781.858105:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.858120:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760781.858125:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.858131:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.858144:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.862632:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986288 is committed 00002000:00010000:2.0:1789760781.863010:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77746176 -> 77750272 00002000:00010000:2.0:1789760781.863016:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151872 -> 151880 00010000:00010000:2.0:1789760781.863030:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.863047:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.863056:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.863063:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.863068:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.863078:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.863098:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.863105:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f0f0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77746176->77750271] (req 77746176->77750271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.863236:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503817856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5186us (5497us total) trans 4294986288 rc 0/0 00000100:00100000:2.0:1789760781.863247:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18992 00000100:00100000:2.0:1789760781.863258:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711972106921 time 711982639732 state 0x2 ref 0 00000100:00100000:2.0:1789760781.868274:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.868429:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503817984 00000100:00100000:0.0:1789760781.868440:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503817984 00000100:00100000:0.0:1789760781.868556:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711972106921 time 711982639732 state 0x0 ref 1 00000100:00100000:0.0:1789760781.868581:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711982900148 time 711982639732 state 0x1 ref 1 00000100:00100000:0.0:1789760781.868652:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711982900148 time 711993258207 state 0x0 ref 1 00000100:00100000:0.0:1789760781.868666:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.868715:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77750272, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25148 00010000:00010000:0.0:1789760781.868725:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.868741:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760781.868745:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.868753:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.868765:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.876067:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986289 is committed 00002000:00010000:0.0:1789760781.876749:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77750272 -> 77754368 00002000:00010000:0.0:1789760781.876755:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151880 -> 151888 00010000:00010000:0.0:1789760781.876768:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.876781:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.876791:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.876799:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.876804:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.876813:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.876840:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.876849:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f0f13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77750272->77754367] (req 77750272->77754367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.876994:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503817984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8325us (8725us total) trans 4294986289 rc 0/0 00000100:00100000:0.0:1789760781.877005:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18993 00000100:00100000:0.0:1789760781.877015:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711982900148 time 711993258207 state 0x2 ref 0 00000100:00100000:1.0:1789760781.883693:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.883862:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503818112 00000100:00100000:2.0:1789760781.883874:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503818112 00000100:00100000:2.0:1789760781.883930:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711982900148 time 711993258207 state 0x0 ref 1 00000100:00100000:2.0:1789760781.883946:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711993518623 time 711993258207 state 0x1 ref 1 00000100:00100000:2.0:1789760781.883980:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711993518623 time 712008586082 state 0x0 ref 1 00000100:00100000:2.0:1789760781.883993:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503818112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.884040:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77754368, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25150 00010000:00010000:2.0:1789760781.884050:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.884066:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760781.884071:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.884077:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.884092:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.889806:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986290 is committed 00002000:00010000:2.0:1789760781.890677:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77754368 -> 77758464 00002000:00010000:2.0:1789760781.890686:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151888 -> 151896 00010000:00010000:2.0:1789760781.890708:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.890732:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.890747:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.890760:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.890770:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.890785:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.890818:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.890831:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77754368->77758463] (req 77754368->77758463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.891667:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503818112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7668us (7976us total) trans 4294986290 rc 0/0 00000100:00100000:2.0:1789760781.891678:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18994 00000100:00100000:2.0:1789760781.891688:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711993518623 time 712008586082 state 0x2 ref 0 00000100:00100000:0.0:1789760781.896944:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.897029:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503818240 00000100:00100000:3.0:1789760781.897038:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503818240 00000100:00100000:3.0:1789760781.897092:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 711993518623 time 712008586082 state 0x0 ref 1 00000100:00100000:3.0:1789760781.897108:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712008846498 time 712008586082 state 0x1 ref 1 00000100:00100000:3.0:1789760781.897140:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712008846498 time 712021746249 state 0x0 ref 1 00000100:00100000:3.0:1789760781.897152:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503818240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.897200:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77758464, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25158 00010000:00010000:3.0:1789760781.897209:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.897226:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760781.897230:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.897237:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.897251:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.901368:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986291 is committed 00002000:00010000:3.0:1789760781.901777:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77758464 -> 77762560 00002000:00010000:3.0:1789760781.901783:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151896 -> 151904 00010000:00010000:3.0:1789760781.901796:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.901810:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.901820:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.901828:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760781.901834:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.901916:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.901937:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.901946:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77758464->77762559] (req 77758464->77762559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.902072:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503818240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4917us (5131us total) trans 4294986291 rc 0/0 00000100:00100000:3.0:1789760781.902084:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18995 00000100:00100000:3.0:1789760781.902094:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712008846498 time 712021746249 state 0x2 ref 0 00000100:00100000:3.0:1789760781.906621:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.907005:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503818368 00000100:00100000:2.0:1789760781.907016:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503818368 00000100:00100000:2.0:1789760781.907072:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712008846498 time 712021746249 state 0x0 ref 1 00000100:00100000:2.0:1789760781.907087:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712022006665 time 712021746249 state 0x1 ref 1 00000100:00100000:2.0:1789760781.907121:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712022006665 time 712031727354 state 0x0 ref 1 00000100:00100000:2.0:1789760781.907134:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503818368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.907179:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77762560, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25160 00010000:00010000:2.0:1789760781.907188:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.907204:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760781.907208:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.907215:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.907229:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.912233:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986292 is committed 00002000:00010000:2.0:1789760781.912379:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77762560 -> 77766656 00002000:00010000:2.0:1789760781.912384:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151904 -> 151912 00010000:00010000:2.0:1789760781.912413:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.912427:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.912437:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.912450:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.912454:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.912463:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.912481:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.912488:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77762560->77766655] (req 77762560->77766655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.912702:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503818368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5565us (6093us total) trans 4294986292 rc 0/0 00000100:00100000:2.0:1789760781.912712:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18996 00000100:00100000:2.0:1789760781.912722:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712022006665 time 712031727354 state 0x2 ref 0 00000100:00100000:1.0:1789760781.917203:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.917336:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503818496 00000100:00100000:0.0:1789760781.917346:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503818496 00000100:00100000:0.0:1789760781.917437:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712022006665 time 712031727354 state 0x0 ref 1 00000100:00100000:0.0:1789760781.917454:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712031987770 time 712031727354 state 0x1 ref 1 00000100:00100000:0.0:1789760781.917524:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712031987770 time 712042130015 state 0x0 ref 1 00000100:00100000:0.0:1789760781.917538:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503818496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.917597:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77766656, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25168 00010000:00010000:0.0:1789760781.917772:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.917794:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760781.917799:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.917806:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.917820:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.923477:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986293 is committed 00002000:00010000:0.0:1789760781.923635:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77766656 -> 77770752 00002000:00010000:0.0:1789760781.923640:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151912 -> 151920 00010000:00010000:0.0:1789760781.923652:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.923664:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.923671:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.923678:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.923683:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.923690:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.923714:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.923722:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77766656->77770751] (req 77766656->77770751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.923813:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503818496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6274us (6616us total) trans 4294986293 rc 0/0 00000100:00100000:0.0:1789760781.923824:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18997 00000100:00100000:0.0:1789760781.923838:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712031987770 time 712042130015 state 0x2 ref 0 00000100:00100000:3.0:1789760781.927129:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.927198:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503818624 00000100:00100000:1.0:1789760781.927208:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503818624 00000100:00100000:1.0:1789760781.927262:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712031987770 time 712042130015 state 0x0 ref 1 00000100:00100000:1.0:1789760781.927277:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712042390431 time 712042130015 state 0x1 ref 1 00000100:00100000:1.0:1789760781.927319:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712042390431 time 712051925431 state 0x0 ref 1 00000100:00100000:1.0:1789760781.927332:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503818624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.927383:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77770752, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25170 00010000:00010000:1.0:1789760781.927431:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.927447:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760781.927451:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.927457:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.927469:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.933625:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986294 is committed 00002000:00010000:1.0:1789760781.933764:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77770752 -> 77774848 00002000:00010000:1.0:1789760781.933770:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151920 -> 151928 00010000:00010000:1.0:1789760781.933783:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.933795:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.933804:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.933813:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.933818:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.933827:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.933845:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.933852:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77770752->77774847] (req 77770752->77774847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.933956:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503818624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6612us (6823us total) trans 4294986294 rc 0/0 00000100:00100000:1.0:1789760781.933967:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18998 00000100:00100000:1.0:1789760781.933982:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712042390431 time 712051925431 state 0x2 ref 0 00000100:00100000:0.0:1789760781.939863:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760781.940001:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503818752 00000100:00100000:3.0:1789760781.940080:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503818752 00000100:00100000:3.0:1789760781.940138:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712042390431 time 712051925431 state 0x0 ref 1 00000100:00100000:3.0:1789760781.940154:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712052185847 time 712051925431 state 0x1 ref 1 00000100:00100000:3.0:1789760781.940196:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712052185847 time 712064802317 state 0x0 ref 1 00000100:00100000:3.0:1789760781.940209:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503818752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760781.940260:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77774848, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25178 00010000:00010000:3.0:1789760781.940270:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.940290:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760781.940295:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.940313:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.940328:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.944813:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986295 is committed 00002000:00010000:0.0:1789760781.944907:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77774848 -> 77778944 00002000:00010000:0.0:1789760781.944912:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151928 -> 151936 00010000:00010000:0.0:1789760781.944926:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.944938:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.944947:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.944955:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.944960:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.944970:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.944990:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.944999:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f0f3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77774848->77778943] (req 77774848->77778943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.945112:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503818752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4901us (5253us total) trans 4294986295 rc 0/0 00000100:00100000:0.0:1789760781.945123:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 18999 00000100:00100000:0.0:1789760781.945133:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712052185847 time 712064802317 state 0x2 ref 0 00000100:00100000:3.0:1789760781.950864:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760781.951156:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503818880 00000100:00100000:2.0:1789760781.951167:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503818880 00000100:00100000:2.0:1789760781.951225:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712052185847 time 712064802317 state 0x0 ref 1 00000100:00100000:2.0:1789760781.951240:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712065062733 time 712064802317 state 0x1 ref 1 00000100:00100000:2.0:1789760781.951273:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712065062733 time 712075879101 state 0x0 ref 1 00000100:00100000:2.0:1789760781.951297:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503818880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760781.951346:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77778944, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25180 00010000:00010000:2.0:1789760781.951355:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.951372:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760781.951377:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.951384:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.951431:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.955429:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986296 is committed 00002000:00010000:0.0:1789760781.955531:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77778944 -> 77783040 00002000:00010000:0.0:1789760781.955537:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151936 -> 151944 00010000:00010000:0.0:1789760781.955549:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.955560:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.955648:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.955661:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.955665:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.955674:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.955694:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.955702:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f0f44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77778944->77783039] (req 77778944->77783039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.955935:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503818880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4643us (5074us total) trans 4294986296 rc 0/0 00000100:00100000:0.0:1789760781.955945:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19000 00000100:00100000:0.0:1789760781.955955:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712065062733 time 712075879101 state 0x2 ref 0 00000100:00100000:3.0:1789760781.960373:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.960472:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819008 00000100:00100000:0.0:1789760781.960483:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819008 00000100:00100000:0.0:1789760781.960539:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712065062733 time 712075879101 state 0x0 ref 1 00000100:00100000:0.0:1789760781.960555:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712076139517 time 712075879101 state 0x1 ref 1 00000100:00100000:0.0:1789760781.960598:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712076139517 time 712085203996 state 0x0 ref 1 00000100:00100000:0.0:1789760781.960613:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503819008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.960661:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77783040, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25188 00010000:00010000:0.0:1789760781.960798:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.960815:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760781.960820:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.960827:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.960842:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.965369:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986297 is committed 00002000:00010000:0.0:1789760781.965638:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77783040 -> 77787136 00002000:00010000:0.0:1789760781.965643:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151944 -> 151952 00010000:00010000:0.0:1789760781.965657:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.965670:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.965679:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.965687:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760781.965693:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.965702:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.965723:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.965731:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0f4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77783040->77787135] (req 77783040->77787135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.965978:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503819008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5362us (5609us total) trans 4294986297 rc 0/0 00000100:00100000:0.0:1789760781.965989:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19001 00000100:00100000:0.0:1789760781.965999:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712076139517 time 712085203996 state 0x2 ref 0 00000100:00100000:2.0:1789760781.972281:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.972712:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819136 00000100:00100000:1.0:1789760781.972722:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819136 00000100:00100000:1.0:1789760781.972775:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712076139517 time 712085203996 state 0x0 ref 1 00000100:00100000:1.0:1789760781.972792:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712085464412 time 712085203996 state 0x1 ref 1 00000100:00100000:1.0:1789760781.972819:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712085464412 time 712097426004 state 0x0 ref 1 00000100:00100000:1.0:1789760781.972828:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503819136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.972868:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77787136, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25190 00010000:00010000:1.0:1789760781.972877:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.972892:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760781.972896:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.972903:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.973119:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.977278:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986298 is committed 00002000:00010000:2.0:1789760781.977700:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77787136 -> 77791232 00002000:00010000:2.0:1789760781.977706:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151952 -> 151960 00010000:00010000:2.0:1789760781.977720:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.977733:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.977743:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.977750:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760781.977755:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.977765:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.977784:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.977792:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f0f52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77787136->77791231] (req 77787136->77791231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.977882:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503819136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5050us (5606us total) trans 4294986298 rc 0/0 00000100:00100000:2.0:1789760781.977892:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19002 00000100:00100000:2.0:1789760781.977902:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712085464412 time 712097426004 state 0x2 ref 0 00000100:00100000:3.0:1789760781.984817:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760781.984993:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819264 00000100:00100000:1.0:1789760781.985003:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819264 00000100:00100000:1.0:1789760781.985056:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712085464412 time 712097426004 state 0x0 ref 1 00000100:00100000:1.0:1789760781.985072:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712097686420 time 712097426004 state 0x1 ref 1 00000100:00100000:1.0:1789760781.985105:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712097686420 time 712109711463 state 0x0 ref 1 00000100:00100000:1.0:1789760781.985118:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503819264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760781.985169:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77791232, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25198 00010000:00010000:1.0:1789760781.985178:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.985196:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760781.985201:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.985207:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.985221:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.989886:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986299 is committed 00002000:00010000:1.0:1789760781.990175:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77791232 -> 77795328 00002000:00010000:1.0:1789760781.990181:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151960 -> 151968 00010000:00010000:1.0:1789760781.990194:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.990208:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.990217:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.990224:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760781.990229:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.990238:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.990257:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.990265:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77791232->77795327] (req 77791232->77795327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.990549:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503819264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5428us (5736us total) trans 4294986299 rc 0/0 00000100:00100000:1.0:1789760781.990560:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19003 00000100:00100000:1.0:1789760781.990581:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712097686420 time 712109711463 state 0x2 ref 0 00000100:00100000:3.0:1789760781.993801:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760781.994186:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819392 00000100:00100000:0.0:1789760781.994196:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819392 00000100:00100000:0.0:1789760781.994246:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712097686420 time 712109711463 state 0x0 ref 1 00000100:00100000:0.0:1789760781.994259:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712109971879 time 712109711463 state 0x1 ref 1 00000100:00100000:0.0:1789760781.997222:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712109971879 time 712121815348 state 0x0 ref 1 00000100:00100000:0.0:1789760781.997256:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503819392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760781.997318:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77795328, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251a0 00010000:00010000:0.0:1789760781.997329:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.997346:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760781.997350:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.997356:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.997372:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.001262:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986300 is committed 00002000:00010000:0.0:1789760782.003762:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77795328 -> 77799424 00002000:00010000:0.0:1789760782.003770:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151968 -> 151976 00010000:00010000:0.0:1789760782.003783:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.003796:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.003805:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.003812:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.003816:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.003825:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.003843:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.003850:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77795328->77799423] (req 77795328->77799423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.003994:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503819392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6747us (10196us total) trans 4294986300 rc 0/0 00000100:00100000:0.0:1789760782.004004:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19004 00000100:00100000:0.0:1789760782.004014:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712109971879 time 712121815348 state 0x2 ref 0 00000100:00100000:0.0:1789760782.007822:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.007909:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819520 00000100:00100000:3.0:1789760782.007920:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819520 00000100:00100000:3.0:1789760782.007988:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712109971879 time 712121815348 state 0x0 ref 1 00000100:00100000:3.0:1789760782.008003:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712122075764 time 712121815348 state 0x1 ref 1 00000100:00100000:3.0:1789760782.008024:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712122075764 time 712132630650 state 0x0 ref 1 00000100:00100000:3.0:1789760782.008036:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503819520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.008095:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77799424, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251a8 00010000:00010000:3.0:1789760782.008104:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.008120:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760782.008125:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.008132:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.008145:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760782.008445:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760782.012671:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986301 is committed 00002000:00010000:3.0:1789760782.012860:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77799424 -> 77803520 00002000:00010000:3.0:1789760782.012865:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151976 -> 151984 00010000:00010000:3.0:1789760782.012878:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.012891:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.012900:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.012950:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.012955:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.012965:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.012985:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.012999:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f0f67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77799424->77803519] (req 77799424->77803519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.013161:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503819520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5122us (5344us total) trans 4294986301 rc 0/0 00000100:00100000:3.0:1789760782.013172:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19005 00000100:00100000:3.0:1789760782.013182:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712122075764 time 712132630650 state 0x2 ref 0 00000100:00100000:3.0:1789760782.016011:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.016328:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819648 00000100:00100000:2.0:1789760782.016339:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819648 00000100:00100000:2.0:1789760782.016422:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712122075764 time 712132630650 state 0x0 ref 1 00000100:00100000:2.0:1789760782.016435:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712132891066 time 712132630650 state 0x1 ref 1 00000100:00100000:2.0:1789760782.016452:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712132891066 time 712141060279 state 0x0 ref 1 00000100:00100000:2.0:1789760782.016463:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503819648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.016701:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77803520, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251b0 00010000:00010000:2.0:1789760782.016710:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.016727:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.016731:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.016738:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.016751:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.020439:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986302 is committed 00002000:00010000:0.0:1789760782.020579:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77803520 -> 77807616 00002000:00010000:0.0:1789760782.020585:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151984 -> 151992 00010000:00010000:0.0:1789760782.020596:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.020609:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.020618:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.020624:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.020631:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.020640:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.020657:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.020664:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f0f6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77803520->77807615] (req 77803520->77807615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.020744:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503819648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4277us (4736us total) trans 4294986302 rc 0/0 00000100:00100000:0.0:1789760782.020754:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19006 00000100:00100000:0.0:1789760782.020764:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712132891066 time 712141060279 state 0x2 ref 0 00000100:00100000:2.0:1789760782.026518:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.026624:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819776 00000100:00100000:1.0:1789760782.026635:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819776 00000100:00100000:1.0:1789760782.026689:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712132891066 time 712141060279 state 0x0 ref 1 00000100:00100000:1.0:1789760782.026704:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712141320695 time 712141060279 state 0x1 ref 1 00000100:00100000:1.0:1789760782.026735:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712141320695 time 712151341470 state 0x0 ref 1 00000100:00100000:1.0:1789760782.026748:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503819776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.026791:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77807616, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251b8 00010000:00010000:1.0:1789760782.026801:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.026817:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.026821:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.026827:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.026840:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.032123:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986303 is committed 00002000:00010000:1.0:1789760782.032690:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77807616 -> 77811712 00002000:00010000:1.0:1789760782.032697:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 151992 -> 152000 00010000:00010000:1.0:1789760782.032710:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.032724:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.032739:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.032747:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.032752:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.032762:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.032782:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.032789:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77807616->77811711] (req 77807616->77811711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.032882:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503819776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6131us (6368us total) trans 4294986303 rc 0/0 00000100:00100000:1.0:1789760782.032892:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19007 00000100:00100000:1.0:1789760782.032901:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712141320695 time 712151341470 state 0x2 ref 0 00000100:00100000:0.0:1789760782.035724:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ccdfdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523610624:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760782.035796:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.036579:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523610624 00000100:00100000:1.0:1789760782.036592:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523610624 00000100:00100000:1.0:1789760782.036648:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:1.0:1789760782.036658:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3639:x1876699523610624:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760782.036697:0:6518:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:1.0:1789760782.036762:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3639:x1876699523610624:12345-0@lo:41:osp-pre-0-1.0 Request processed in 104us (964us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.036770:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789760782.038799:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccdfdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523610624:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760782.041599:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.041858:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503819904 00000100:00100000:0.0:1789760782.041868:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503819904 00000100:00100000:0.0:1789760782.041985:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712141320695 time 712151341470 state 0x0 ref 1 00000100:00100000:0.0:1789760782.042000:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712151601886 time 712151341470 state 0x1 ref 1 00000100:00100000:0.0:1789760782.042062:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712151601886 time 712166667745 state 0x0 ref 1 00000100:00100000:0.0:1789760782.042136:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503819904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.042182:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77811712, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251c0 00010000:00010000:0.0:1789760782.042191:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.042206:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.042211:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.042217:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.042776:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.050983:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986304 is committed 00002000:00010000:0.0:1789760782.051207:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77811712 -> 77815808 00002000:00010000:0.0:1789760782.051213:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152000 -> 152008 00010000:00010000:0.0:1789760782.051225:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.051238:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.051247:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.051255:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.051260:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.051269:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.051347:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.051356:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77811712->77815807] (req 77811712->77815807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.051582:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503819904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9492us (9976us total) trans 4294986304 rc 0/0 00000100:00100000:0.0:1789760782.051592:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19008 00000100:00100000:0.0:1789760782.051603:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712151601886 time 712166667745 state 0x2 ref 0 00000100:00100000:3.0:1789760782.055885:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.056452:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820032 00000100:00100000:0.0:1789760782.056463:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820032 00000100:00100000:0.0:1789760782.056525:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712151601886 time 712166667745 state 0x0 ref 1 00000100:00100000:0.0:1789760782.056541:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712166928161 time 712166667745 state 0x1 ref 1 00000100:00100000:0.0:1789760782.056940:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712166928161 time 712181544763 state 0x0 ref 1 00000100:00100000:0.0:1789760782.057147:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503820032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.057193:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77815808, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251c8 00010000:00010000:0.0:1789760782.057203:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.057219:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.057223:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.057230:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.057244:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.064235:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986305 is committed 00002000:00010000:0.0:1789760782.064642:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77815808 -> 77819904 00002000:00010000:0.0:1789760782.064648:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152008 -> 152016 00010000:00010000:0.0:1789760782.064660:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.064672:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.064681:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.064688:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.064692:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.064701:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.064720:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.064728:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77815808->77819903] (req 77815808->77819903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.064868:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503820032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7722us (8988us total) trans 4294986305 rc 0/0 00000100:00100000:0.0:1789760782.064879:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19009 00000100:00100000:0.0:1789760782.064889:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712166928161 time 712181544763 state 0x2 ref 0 00000100:00100000:2.0:1789760782.069840:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.070053:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820160 00000100:00100000:0.0:1789760782.070062:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820160 00000100:00100000:0.0:1789760782.070114:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712166928161 time 712181544763 state 0x0 ref 1 00000100:00100000:0.0:1789760782.070129:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712181805179 time 712181544763 state 0x1 ref 1 00000100:00100000:0.0:1789760782.070198:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712181805179 time 712194803497 state 0x0 ref 1 00000100:00100000:0.0:1789760782.070211:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503820160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.070257:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77819904, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251d0 00010000:00010000:0.0:1789760782.070267:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.070283:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.070289:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.070307:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.070320:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.076072:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986306 is committed 00002000:00010000:0.0:1789760782.076195:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77819904 -> 77824000 00002000:00010000:0.0:1789760782.076200:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152016 -> 152024 00010000:00010000:0.0:1789760782.076213:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.076226:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.076235:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.076243:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.076248:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.076256:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.076275:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.076292:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0f8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77819904->77823999] (req 77819904->77823999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.076382:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503820160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6169us (6547us total) trans 4294986306 rc 0/0 00000100:00100000:0.0:1789760782.076439:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19010 00000100:00100000:0.0:1789760782.076450:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712181805179 time 712194803497 state 0x2 ref 0 00000100:00100000:2.0:1789760782.081742:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.081968:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820288 00000100:00100000:1.0:1789760782.081977:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820288 00000100:00100000:1.0:1789760782.082042:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712181805179 time 712194803497 state 0x0 ref 1 00000100:00100000:1.0:1789760782.082057:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712195063913 time 712194803497 state 0x1 ref 1 00000100:00100000:1.0:1789760782.082094:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712195063913 time 712206699627 state 0x0 ref 1 00000100:00100000:1.0:1789760782.082108:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.082156:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77824000, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251d8 00010000:00010000:1.0:1789760782.082166:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.082181:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.082186:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.082192:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.082205:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.089110:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986307 is committed 00002000:00010000:1.0:1789760782.089234:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77824000 -> 77828096 00002000:00010000:1.0:1789760782.089240:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152024 -> 152032 00010000:00010000:1.0:1789760782.089253:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.089645:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.089658:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.089665:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.089669:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.089678:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.089697:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.089706:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0f91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77824000->77828095] (req 77824000->77828095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.089799:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7689us (8061us total) trans 4294986307 rc 0/0 00000100:00100000:1.0:1789760782.089810:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19011 00000100:00100000:1.0:1789760782.089820:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712195063913 time 712206699627 state 0x2 ref 0 00000100:00100000:3.0:1789760782.094346:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.094381:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820416 00000100:00100000:2.0:1789760782.094425:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820416 00000100:00100000:2.0:1789760782.094481:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712195063913 time 712206699627 state 0x0 ref 1 00000100:00100000:2.0:1789760782.094497:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712206960043 time 712206699627 state 0x1 ref 1 00000100:00100000:2.0:1789760782.094528:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712206960043 time 712219134146 state 0x0 ref 1 00000100:00100000:2.0:1789760782.094542:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503820416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.094656:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77828096, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251e0 00010000:00010000:2.0:1789760782.094669:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.094685:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.094689:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.094695:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.094709:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.100718:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986308 is committed 00002000:00010000:2.0:1789760782.100833:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77828096 -> 77832192 00002000:00010000:2.0:1789760782.100839:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152032 -> 152040 00010000:00010000:2.0:1789760782.100852:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.100877:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.100886:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.100893:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.100899:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.100908:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.100929:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.100936:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0f98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77828096->77832191] (req 77828096->77832191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.101040:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503820416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6495us (6697us total) trans 4294986308 rc 0/0 00000100:00100000:2.0:1789760782.101050:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19012 00000100:00100000:2.0:1789760782.101061:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712206960043 time 712219134146 state 0x2 ref 0 00000100:00100000:2.0:1789760782.105535:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.105626:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820544 00000100:00100000:0.0:1789760782.105637:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820544 00000100:00100000:0.0:1789760782.105697:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712206960043 time 712219134146 state 0x0 ref 1 00000100:00100000:0.0:1789760782.105711:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712219394562 time 712219134146 state 0x1 ref 1 00000100:00100000:0.0:1789760782.105742:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712219394562 time 712230348873 state 0x0 ref 1 00000100:00100000:0.0:1789760782.105754:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.105799:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77832192, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251e8 00010000:00010000:0.0:1789760782.105808:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.105824:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.105828:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.105834:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.105853:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.110641:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986309 is committed 00002000:00010000:0.0:1789760782.111022:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77832192 -> 77836288 00002000:00010000:0.0:1789760782.111029:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152040 -> 152048 00010000:00010000:0.0:1789760782.111043:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.111057:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.111066:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.111074:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.111079:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.111088:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.111108:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.111117:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0f9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77832192->77836287] (req 77832192->77836287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.111508:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5749us (5981us total) trans 4294986309 rc 0/0 00000100:00100000:0.0:1789760782.111520:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19013 00000100:00100000:0.0:1789760782.111529:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712219394562 time 712230348873 state 0x2 ref 0 00000100:00100000:0.0:1789760782.116725:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.117592:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820672 00000100:00100000:2.0:1789760782.117603:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820672 00000100:00100000:2.0:1789760782.117724:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712219394562 time 712230348873 state 0x0 ref 1 00000100:00100000:2.0:1789760782.117740:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712230609289 time 712230348873 state 0x1 ref 1 00000100:00100000:2.0:1789760782.117816:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712230609289 time 712242421921 state 0x0 ref 1 00000100:00100000:2.0:1789760782.117830:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.117879:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77836288, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251f0 00010000:00010000:2.0:1789760782.117889:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.117905:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.117910:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.117927:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.117942:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.123927:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986310 is committed 00002000:00010000:2.0:1789760782.124044:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77836288 -> 77840384 00002000:00010000:2.0:1789760782.124049:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152048 -> 152056 00010000:00010000:2.0:1789760782.124061:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.124142:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.124152:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.124159:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.124164:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.124178:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.124196:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.124202:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f0fa6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77836288->77840383] (req 77836288->77840383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.124302:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6460us (7573us total) trans 4294986310 rc 0/0 00000100:00100000:2.0:1789760782.124313:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19014 00000100:00100000:2.0:1789760782.124324:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712230609289 time 712242421921 state 0x2 ref 0 00000100:00100000:3.0:1789760782.128939:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.129338:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820800 00000100:00100000:1.0:1789760782.129350:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820800 00000100:00100000:1.0:1789760782.129489:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712230609289 time 712242421921 state 0x0 ref 1 00000100:00100000:1.0:1789760782.129504:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712242682337 time 712242421921 state 0x1 ref 1 00000100:00100000:1.0:1789760782.129579:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712242682337 time 712254184669 state 0x0 ref 1 00000100:00100000:1.0:1789760782.129592:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.129645:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77840384, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x251f8 00010000:00010000:1.0:1789760782.129655:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.129676:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.129682:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.129688:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.129701:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.136503:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986311 is committed 00002000:00010000:0.0:1789760782.140159:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77840384 -> 77844480 00002000:00010000:0.0:1789760782.140169:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152056 -> 152064 00010000:00010000:0.0:1789760782.140182:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.140195:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.140203:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.140210:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.140215:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.140224:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.140243:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.140252:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f0fad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77840384->77844479] (req 77840384->77844479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.141107:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503820800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11508us (12167us total) trans 4294986311 rc 0/0 00000100:00100000:0.0:1789760782.141121:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19015 00000100:00100000:0.0:1789760782.141135:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712242682337 time 712254184669 state 0x2 ref 0 00000100:00100000:3.0:1789760782.145102:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.145760:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503820928 00000100:00100000:0.0:1789760782.145770:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503820928 00000100:00100000:0.0:1789760782.145831:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712242682337 time 712254184669 state 0x0 ref 1 00000100:00100000:0.0:1789760782.145847:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712254445085 time 712254184669 state 0x1 ref 1 00000100:00100000:0.0:1789760782.145943:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712254445085 time 712270548450 state 0x0 ref 1 00000100:00100000:0.0:1789760782.145956:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503820928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.146001:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77844480, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25200 00010000:00010000:0.0:1789760782.146009:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.146024:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.146029:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.146034:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.146052:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.154443:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986312 is committed 00002000:00010000:0.0:1789760782.154583:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77844480 -> 77848576 00002000:00010000:0.0:1789760782.154589:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152064 -> 152072 00010000:00010000:0.0:1789760782.154603:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154616:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154626:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154633:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.154638:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154646:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154665:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154674:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77844480->77848575] (req 77844480->77848575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.154780:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503820928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8822us (9683us total) trans 4294986312 rc 0/0 00000100:00100000:0.0:1789760782.154791:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19016 00000100:00100000:0.0:1789760782.154800:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712254445085 time 712270548450 state 0x2 ref 0 00000100:00100000:2.0:1789760782.158426:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.158518:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821056 00000100:00100000:0.0:1789760782.158529:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821056 00000100:00100000:0.0:1789760782.158595:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712254445085 time 712270548450 state 0x0 ref 1 00000100:00100000:0.0:1789760782.158610:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712270808866 time 712270548450 state 0x1 ref 1 00000100:00100000:0.0:1789760782.158633:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712270808866 time 712283240023 state 0x0 ref 1 00000100:00100000:0.0:1789760782.158645:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.158693:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77848576, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25208 00010000:00010000:0.0:1789760782.158701:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.158717:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.158721:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.158728:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.158741:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.165253:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986313 is committed 00002000:00010000:0.0:1789760782.165362:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77848576 -> 77852672 00002000:00010000:0.0:1789760782.165368:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152072 -> 152080 00010000:00010000:0.0:1789760782.165380:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.165433:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.165443:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.165450:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.165455:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.165464:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.165484:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.165491:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f0fbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77848576->77852671] (req 77848576->77852671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.168685:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10030us (10288us total) trans 4294986313 rc 0/0 00000100:00100000:0.0:1789760782.168701:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19017 00000100:00100000:0.0:1789760782.168711:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712270808866 time 712283240023 state 0x2 ref 0 00000100:00100000:3.0:1789760782.185742:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.190738:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821184 00000100:00100000:2.0:1789760782.190754:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821184 00000100:00100000:2.0:1789760782.190818:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712270808866 time 712283240023 state 0x0 ref 1 00000100:00100000:2.0:1789760782.190834:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712283500439 time 712283240023 state 0x1 ref 1 00000100:00100000:2.0:1789760782.190869:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712283500439 time 712315475534 state 0x0 ref 1 00000100:00100000:2.0:1789760782.190882:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503821184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.190929:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77852672, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25210 00010000:00010000:2.0:1789760782.190940:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.190955:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.190960:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.190976:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.190989:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.198090:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986314 is committed 00002000:00010000:2.0:1789760782.198218:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77852672 -> 77856768 00002000:00010000:2.0:1789760782.198223:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152080 -> 152088 00010000:00010000:2.0:1789760782.198237:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.198249:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.198258:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.198265:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.198271:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.198281:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.198311:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.198319:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77852672->77856767] (req 77852672->77856767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.198583:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503821184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7697us (12844us total) trans 4294986314 rc 0/0 00000100:00100000:2.0:1789760782.198597:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19018 00000100:00100000:2.0:1789760782.198887:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712283500439 time 712315475534 state 0x2 ref 0 00000100:00100000:3.0:1789760782.204692:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.204795:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821312 00000100:00100000:1.0:1789760782.204806:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821312 00000100:00100000:1.0:1789760782.204860:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712283500439 time 712315475534 state 0x0 ref 1 00000100:00100000:1.0:1789760782.204877:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712315735950 time 712315475534 state 0x1 ref 1 00000100:00100000:1.0:1789760782.204973:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712315735950 time 712329568248 state 0x0 ref 1 00000100:00100000:1.0:1789760782.204987:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503821312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.205034:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77856768, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25218 00010000:00010000:1.0:1789760782.205044:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.205059:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.205064:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.205070:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.205084:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.213183:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986315 is committed 00002000:00010000:1.0:1789760782.213290:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77856768 -> 77860864 00002000:00010000:1.0:1789760782.213295:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152088 -> 152096 00010000:00010000:1.0:1789760782.213309:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.213322:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.213330:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.213336:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.213340:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.213349:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.213369:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.213376:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77856768->77860863] (req 77856768->77860863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.213517:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503821312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8527us (8829us total) trans 4294986315 rc 0/0 00000100:00100000:1.0:1789760782.213529:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19019 00000100:00100000:1.0:1789760782.213539:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712315735950 time 712329568248 state 0x2 ref 0 00000100:00100000:0.0:1789760782.219729:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.219833:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821440 00000100:00100000:2.0:1789760782.219842:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821440 00000100:00100000:2.0:1789760782.219895:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712315735950 time 712329568248 state 0x0 ref 1 00000100:00100000:2.0:1789760782.219909:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712329828664 time 712329568248 state 0x1 ref 1 00000100:00100000:2.0:1789760782.219939:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712329828664 time 712344545759 state 0x0 ref 1 00000100:00100000:2.0:1789760782.219950:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503821440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.220020:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77860864, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25220 00010000:00010000:2.0:1789760782.220030:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.220046:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.220052:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.220057:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.220070:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.225880:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986316 is committed 00002000:00010000:2.0:1789760782.226251:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77860864 -> 77864960 00002000:00010000:2.0:1789760782.226257:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152096 -> 152104 00010000:00010000:2.0:1789760782.226271:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.226284:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.226302:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.226310:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.226315:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.226324:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.226343:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.226357:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77860864->77864959] (req 77860864->77864959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.227243:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503821440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7286us (7643us total) trans 4294986316 rc 0/0 00000100:00100000:2.0:1789760782.227255:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19020 00000100:00100000:2.0:1789760782.227265:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712329828664 time 712344545759 state 0x2 ref 0 00000100:00100000:3.0:1789760782.230447:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.230644:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821568 00000100:00100000:0.0:1789760782.230654:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821568 00000100:00100000:0.0:1789760782.230710:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712329828664 time 712344545759 state 0x0 ref 1 00000100:00100000:0.0:1789760782.230726:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712344806175 time 712344545759 state 0x1 ref 1 00000100:00100000:0.0:1789760782.230757:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712344806175 time 712355363114 state 0x0 ref 1 00000100:00100000:0.0:1789760782.230769:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.230816:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77864960, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25228 00010000:00010000:0.0:1789760782.230828:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.230844:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.230849:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.230855:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.230869:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.242580:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986317 is committed 00002000:00010000:3.0:1789760782.243097:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77864960 -> 77869056 00002000:00010000:3.0:1789760782.243103:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152104 -> 152112 00010000:00010000:3.0:1789760782.243116:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.243128:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.243138:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.243146:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.243152:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.243161:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.243181:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.243189:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f0fd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77864960->77869055] (req 77864960->77869055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.243276:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12503us (12832us total) trans 4294986317 rc 0/0 00000100:00100000:3.0:1789760782.244022:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19021 00000100:00100000:3.0:1789760782.244035:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712344806175 time 712355363114 state 0x2 ref 0 00000100:00100000:3.0:1789760782.246153:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.246432:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821696 00000100:00100000:0.0:1789760782.246441:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821696 00000100:00100000:0.0:1789760782.246495:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712344806175 time 712355363114 state 0x0 ref 1 00000100:00100000:0.0:1789760782.246510:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712355623530 time 712355363114 state 0x1 ref 1 00000100:00100000:0.0:1789760782.246540:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712355623530 time 712371146628 state 0x0 ref 1 00000100:00100000:0.0:1789760782.246552:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.246910:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77869056, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25230 00010000:00010000:0.0:1789760782.246920:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.246935:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.246939:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.246946:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.246960:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.256777:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986318 is committed 00002000:00010000:0.0:1789760782.258895:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77869056 -> 77873152 00002000:00010000:0.0:1789760782.258904:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152112 -> 152120 00010000:00010000:0.0:1789760782.258919:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.258933:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.258942:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.258950:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.258955:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.258963:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.258984:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.258992:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f0fde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77869056->77873151] (req 77869056->77873151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.259183:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12627us (13032us total) trans 4294986318 rc 0/0 00000100:00100000:0.0:1789760782.259193:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19022 00000100:00100000:0.0:1789760782.259202:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712355623530 time 712371146628 state 0x2 ref 0 00000100:00100000:0.0:1789760782.264789:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.264904:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821824 00000100:00100000:1.0:1789760782.264913:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821824 00000100:00100000:1.0:1789760782.264966:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712355623530 time 712371146628 state 0x0 ref 1 00000100:00100000:1.0:1789760782.264980:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712371407044 time 712371146628 state 0x1 ref 1 00000100:00100000:1.0:1789760782.265008:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712371407044 time 712389615202 state 0x0 ref 1 00000100:00100000:1.0:1789760782.265022:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.265066:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77873152, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25238 00010000:00010000:1.0:1789760782.265076:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.265091:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.265157:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.265164:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.265180:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.275756:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986319 is committed 00002000:00010000:1.0:1789760782.275872:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77873152 -> 77877248 00002000:00010000:1.0:1789760782.275877:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152120 -> 152128 00010000:00010000:1.0:1789760782.275890:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.275908:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.275918:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.275925:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.275930:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.275939:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.275958:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.275965:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f0fe5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77873152->77877247] (req 77873152->77877247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.276278:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503821824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11254us (11495us total) trans 4294986319 rc 0/0 00000100:00100000:1.0:1789760782.276301:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19023 00000100:00100000:1.0:1789760782.276311:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712371407044 time 712389615202 state 0x2 ref 0 00000100:00100000:3.0:1789760782.281043:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.281574:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503821952 00000100:00100000:2.0:1789760782.281585:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503821952 00000100:00100000:2.0:1789760782.281707:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712371407044 time 712389615202 state 0x0 ref 1 00000100:00100000:2.0:1789760782.281722:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712389875618 time 712389615202 state 0x1 ref 1 00000100:00100000:2.0:1789760782.281864:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712389875618 time 712406469137 state 0x0 ref 1 00000100:00100000:2.0:1789760782.281878:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503821952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.281926:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77877248, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25240 00010000:00010000:2.0:1789760782.281936:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.281952:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.281956:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.281962:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.281976:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.289153:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986320 is committed 00002000:00010000:2.0:1789760782.289276:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77877248 -> 77881344 00002000:00010000:2.0:1789760782.289281:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152128 -> 152136 00010000:00010000:2.0:1789760782.289443:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.289457:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.289466:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.289472:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.289477:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.289484:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.289505:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.289512:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0fec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77877248->77881343] (req 77877248->77881343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.289615:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503821952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7725us (8566us total) trans 4294986320 rc 0/0 00000100:00100000:2.0:1789760782.289626:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19024 00000100:00100000:2.0:1789760782.289636:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712389875618 time 712406469137 state 0x2 ref 0 00000100:00100000:3.0:1789760782.292730:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.293074:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822080 00000100:00100000:0.0:1789760782.293084:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822080 00000100:00100000:0.0:1789760782.293136:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712389875618 time 712406469137 state 0x0 ref 1 00000100:00100000:0.0:1789760782.293150:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712406729553 time 712406469137 state 0x1 ref 1 00000100:00100000:0.0:1789760782.293184:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712406729553 time 712417790771 state 0x0 ref 1 00000100:00100000:0.0:1789760782.293195:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.293240:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77881344, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25248 00010000:00010000:0.0:1789760782.293248:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.293263:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.293267:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.293273:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.293536:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.299688:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986321 is committed 00002000:00010000:0.0:1789760782.299811:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77881344 -> 77885440 00002000:00010000:0.0:1789760782.299817:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152136 -> 152144 00010000:00010000:0.0:1789760782.299830:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.299844:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.299854:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.299861:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.299866:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.299876:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.299896:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.299905:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f0ff3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77881344->77885439] (req 77881344->77885439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.299990:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6793us (7264us total) trans 4294986321 rc 0/0 00000100:00100000:0.0:1789760782.300000:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19025 00000100:00100000:0.0:1789760782.300009:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712406729553 time 712417790771 state 0x2 ref 0 00000100:00100000:0.0:1789760782.307554:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.307798:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822208 00000100:00100000:2.0:1789760782.307808:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822208 00000100:00100000:2.0:1789760782.307863:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712406729553 time 712417790771 state 0x0 ref 1 00000100:00100000:2.0:1789760782.307878:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712418051187 time 712417790771 state 0x1 ref 1 00000100:00100000:2.0:1789760782.307948:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712418051187 time 712432554030 state 0x0 ref 1 00000100:00100000:2.0:1789760782.307962:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.308009:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77885440, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25250 00010000:00010000:2.0:1789760782.308019:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.308035:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.308039:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.308045:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.308057:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.314558:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986322 is committed 00002000:00010000:2.0:1789760782.314838:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77885440 -> 77889536 00002000:00010000:2.0:1789760782.314843:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152144 -> 152152 00010000:00010000:2.0:1789760782.314856:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.314868:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.314876:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.314882:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.314887:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.314895:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.314912:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.314919:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f0ffa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77885440->77889535] (req 77885440->77889535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.315005:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7041us (7456us total) trans 4294986322 rc 0/0 00000100:00100000:2.0:1789760782.315015:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19026 00000100:00100000:2.0:1789760782.315025:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712418051187 time 712432554030 state 0x2 ref 0 00000100:00100000:3.0:1789760782.320345:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.320464:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822336 00000100:00100000:2.0:1789760782.320473:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822336 00000100:00100000:2.0:1789760782.320526:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712418051187 time 712432554030 state 0x0 ref 1 00000100:00100000:2.0:1789760782.320542:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712432814446 time 712432554030 state 0x1 ref 1 00000100:00100000:2.0:1789760782.320572:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712432814446 time 712445169351 state 0x0 ref 1 00000100:00100000:2.0:1789760782.320585:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.320694:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77889536, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25258 00010000:00010000:2.0:1789760782.320705:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.320719:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.320723:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.320729:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.320742:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.326855:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986323 is committed 00002000:00010000:2.0:1789760782.327092:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77889536 -> 77893632 00002000:00010000:2.0:1789760782.327098:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152152 -> 152160 00010000:00010000:2.0:1789760782.327121:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.327134:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.327144:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.327152:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.327157:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.327167:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.327186:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.327192:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1001 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77889536->77893631] (req 77889536->77893631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.327558:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6969us (7217us total) trans 4294986323 rc 0/0 00000100:00100000:2.0:1789760782.327580:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19027 00000100:00100000:2.0:1789760782.327590:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712432814446 time 712445169351 state 0x2 ref 0 00000100:00100000:3.0:1789760782.334843:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.339461:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822464 00000100:00100000:0.0:1789760782.339478:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822464 00000100:00100000:0.0:1789760782.339545:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712432814446 time 712445169351 state 0x0 ref 1 00000100:00100000:0.0:1789760782.339561:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712445429767 time 712445169351 state 0x1 ref 1 00000100:00100000:2.0:1789760782.339660:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712445429767 time 712464264041 state 0x0 ref 1 00000100:00100000:2.0:1789760782.339681:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.339729:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77893632, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25260 00010000:00010000:2.0:1789760782.339738:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339758:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.339763:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.339769:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.339785:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.347453:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986324 is committed 00002000:00010000:2.0:1789760782.347864:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77893632 -> 77897728 00002000:00010000:2.0:1789760782.347871:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152160 -> 152168 00010000:00010000:2.0:1789760782.347886:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.347899:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.347908:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.347920:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.347925:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.347935:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.347955:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.347962:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1008 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77893632->77897727] (req 77893632->77897727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.348106:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8425us (13268us total) trans 4294986324 rc 0/0 00000100:00100000:2.0:1789760782.348117:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19028 00000100:00100000:2.0:1789760782.348127:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712445429767 time 712464264041 state 0x2 ref 0 00000100:00100000:0.0:1789760782.363543:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.371228:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822592 00000100:00100000:3.0:1789760782.371242:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822592 00000100:00100000:2.0:1789760782.372530:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712445429767 time 712464264041 state 0x0 ref 1 00000100:00100000:2.0:1789760782.372547:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712464524457 time 712464264041 state 0x1 ref 1 00000100:00100000:2.0:1789760782.372597:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712464524457 time 712497203270 state 0x0 ref 1 00000100:00100000:2.0:1789760782.372613:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.372661:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77897728, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25268 00010000:00010000:2.0:1789760782.372671:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.372687:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.372692:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.372698:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.372712:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.379002:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986325 is committed 00002000:00010000:2.0:1789760782.379117:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77897728 -> 77901824 00002000:00010000:2.0:1789760782.379122:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152168 -> 152176 00010000:00010000:2.0:1789760782.379137:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.379150:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.379158:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.379165:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.379169:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.379183:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.379203:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.379210:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f100f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77897728->77901823] (req 77897728->77901823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.379320:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6706us (15782us total) trans 4294986325 rc 0/0 00000100:00100000:2.0:1789760782.379331:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19029 00000100:00100000:2.0:1789760782.379341:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712464524457 time 712497203270 state 0x2 ref 0 00000100:00100000:2.0:1789760782.387523:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.387887:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822720 00000100:00100000:0.0:1789760782.387896:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822720 00000100:00100000:0.0:1789760782.387961:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712464524457 time 712497203270 state 0x0 ref 1 00000100:00100000:0.0:1789760782.387977:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712497463686 time 712497203270 state 0x1 ref 1 00000100:00100000:0.0:1789760782.388045:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712497463686 time 712512651248 state 0x0 ref 1 00000100:00100000:0.0:1789760782.388057:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.388103:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77901824, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25270 00010000:00010000:0.0:1789760782.388113:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.388138:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.388142:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.388148:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.388163:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.393770:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986326 is committed 00002000:00010000:0.0:1789760782.394129:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77901824 -> 77905920 00002000:00010000:0.0:1789760782.394134:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152176 -> 152184 00010000:00010000:0.0:1789760782.394149:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394229:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394239:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394246:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.394252:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394262:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394281:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394346:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1016 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77901824->77905919] (req 77901824->77905919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.394697:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6636us (7176us total) trans 4294986326 rc 0/0 00000100:00100000:0.0:1789760782.394708:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19030 00000100:00100000:0.0:1789760782.394718:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712497463686 time 712512651248 state 0x2 ref 0 00000100:00100000:2.0:1789760782.405126:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.405269:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822848 00000100:00100000:1.0:1789760782.405280:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822848 00000100:00100000:1.0:1789760782.405476:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712497463686 time 712512651248 state 0x0 ref 1 00000100:00100000:1.0:1789760782.405492:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712512911664 time 712512651248 state 0x1 ref 1 00000100:00100000:1.0:1789760782.405638:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712512911664 time 712530169751 state 0x0 ref 1 00000100:00100000:1.0:1789760782.405653:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.405704:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77905920, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25278 00010000:00010000:1.0:1789760782.405714:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.405729:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.405734:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.405740:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.405763:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.415748:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986327 is committed 00002000:00010000:1.0:1789760782.418106:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77905920 -> 77910016 00002000:00010000:1.0:1789760782.418116:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152184 -> 152192 00010000:00010000:1.0:1789760782.418130:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418149:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418157:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418164:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.418169:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418178:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418197:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418205:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f101d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77905920->77910015] (req 77905920->77910015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.418706:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503822848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13049us (13583us total) trans 4294986327 rc 0/0 00000100:00100000:1.0:1789760782.418719:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19031 00000100:00100000:1.0:1789760782.418729:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712512911664 time 712530169751 state 0x2 ref 0 00000100:00100000:2.0:1789760782.428963:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.429637:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503822976 00000100:00100000:0.0:1789760782.429647:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503822976 00000100:00100000:0.0:1789760782.429703:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712512911664 time 712530169751 state 0x0 ref 1 00000100:00100000:0.0:1789760782.429716:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712530430167 time 712530169751 state 0x1 ref 1 00000100:00100000:0.0:1789760782.429747:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712530430167 time 712554354061 state 0x0 ref 1 00000100:00100000:0.0:1789760782.429768:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.429814:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77910016, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25280 00010000:00010000:0.0:1789760782.429823:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.429839:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.429843:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.429848:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.429861:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.435005:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986328 is committed 00002000:00010000:0.0:1789760782.435294:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77910016 -> 77914112 00002000:00010000:0.0:1789760782.435300:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152192 -> 152200 00010000:00010000:0.0:1789760782.435313:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.435332:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.435342:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.435350:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.435355:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.435363:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.435383:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.435420:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1024 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77910016->77914111] (req 77910016->77914111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.435576:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503822976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5793us (6605us total) trans 4294986328 rc 0/0 00000100:00100000:0.0:1789760782.435589:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19032 00000100:00100000:0.0:1789760782.435607:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712530430167 time 712554354061 state 0x2 ref 0 00000100:00100000:2.0:1789760782.439690:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.439799:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503823104 00000100:00100000:0.0:1789760782.439811:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503823104 00000100:00100000:0.0:1789760782.439933:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712530430167 time 712554354061 state 0x0 ref 1 00000100:00100000:0.0:1789760782.439949:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712554614477 time 712554354061 state 0x1 ref 1 00000100:00100000:0.0:1789760782.440015:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712554614477 time 712564621180 state 0x0 ref 1 00000100:00100000:0.0:1789760782.440028:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503823104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.440080:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77914112, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25288 00010000:00010000:0.0:1789760782.440089:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.440104:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.440109:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.440116:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.440143:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.445364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986329 is committed 00002000:00010000:0.0:1789760782.445584:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77914112 -> 77918208 00002000:00010000:0.0:1789760782.445589:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152200 -> 152208 00010000:00010000:0.0:1789760782.445612:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.445625:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.445640:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.445647:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.445652:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.445661:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.445678:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.445686:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f102b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77914112->77918207] (req 77914112->77918207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.445775:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503823104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5743us (6089us total) trans 4294986329 rc 0/0 00000100:00100000:0.0:1789760782.445785:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19033 00000100:00100000:0.0:1789760782.445795:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712554614477 time 712564621180 state 0x2 ref 0 00000100:00100000:2.0:1789760782.449620:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.449704:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503823232 00000100:00100000:0.0:1789760782.449712:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503823232 00000100:00100000:0.0:1789760782.449765:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712554614477 time 712564621180 state 0x0 ref 1 00000100:00100000:0.0:1789760782.449779:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712564881596 time 712564621180 state 0x1 ref 1 00000100:00100000:0.0:1789760782.449813:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712564881596 time 712574419215 state 0x0 ref 1 00000100:00100000:0.0:1789760782.449826:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503823232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.449871:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77918208, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25290 00010000:00010000:0.0:1789760782.449880:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.449894:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.449899:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.449905:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.449917:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.456800:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986330 is committed 00002000:00010000:0.0:1789760782.459637:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77918208 -> 77922304 00002000:00010000:0.0:1789760782.459646:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152208 -> 152216 00010000:00010000:0.0:1789760782.459659:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.459671:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.459679:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.459692:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.459696:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.459704:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.459723:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.459731:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1032 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77918208->77922303] (req 77918208->77922303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.459835:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503823232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10007us (10229us total) trans 4294986330 rc 0/0 00000100:00100000:0.0:1789760782.459845:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19034 00000100:00100000:0.0:1789760782.459854:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712564881596 time 712574419215 state 0x2 ref 0 00000100:00100000:2.0:1789760782.472015:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.472102:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503823360 00000100:00100000:1.0:1789760782.472112:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503823360 00000100:00100000:1.0:1789760782.472178:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712564881596 time 712574419215 state 0x0 ref 1 00000100:00100000:1.0:1789760782.472195:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712574679631 time 712574419215 state 0x1 ref 1 00000100:00100000:1.0:1789760782.472229:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712574679631 time 712596835824 state 0x0 ref 1 00000100:00100000:1.0:1789760782.472241:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503823360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.472285:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77922304, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25298 00010000:00010000:1.0:1789760782.472306:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.472320:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.472324:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.472330:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.472344:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.483437:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986331 is committed 00002000:00010000:1.0:1789760782.483971:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77922304 -> 77926400 00002000:00010000:1.0:1789760782.483977:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152216 -> 152224 00010000:00010000:1.0:1789760782.483992:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.484005:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.484014:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.484021:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.484026:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.484035:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.484055:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.484063:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1039 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77922304->77926399] (req 77922304->77926399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.484766:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503823360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12511us (12745us total) trans 4294986331 rc 0/0 00000100:00100000:1.0:1789760782.484778:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19035 00000100:00100000:1.0:1789760782.484788:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712574679631 time 712596835824 state 0x2 ref 0 00000100:00100000:2.0:1789760782.494498:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.494580:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503823488 00000100:00100000:0.0:1789760782.494590:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503823488 00000100:00100000:0.0:1789760782.494653:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712574679631 time 712596835824 state 0x0 ref 1 00000100:00100000:0.0:1789760782.494668:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712597096240 time 712596835824 state 0x1 ref 1 00000100:00100000:0.0:1789760782.494700:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712597096240 time 712619306113 state 0x0 ref 1 00000100:00100000:0.0:1789760782.494714:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503823488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.494759:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77926400, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252a0 00010000:00010000:0.0:1789760782.494767:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.494782:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.494788:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.494794:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.494808:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.500767:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986332 is committed 00002000:00010000:0.0:1789760782.500891:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77926400 -> 77930496 00002000:00010000:0.0:1789760782.500896:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152224 -> 152232 00010000:00010000:0.0:1789760782.500908:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.500921:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.500929:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.500936:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.500941:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.500950:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.500969:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.500977:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1040 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77926400->77930495] (req 77926400->77930495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.501069:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503823488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6352us (6575us total) trans 4294986332 rc 0/0 00000100:00100000:0.0:1789760782.501080:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19036 00000100:00100000:0.0:1789760782.501089:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712597096240 time 712619306113 state 0x2 ref 0 00000100:00100000:2.0:1789760782.507038:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.507120:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503823616 00000100:00100000:0.0:1789760782.507131:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503823616 00000100:00100000:0.0:1789760782.507196:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712597096240 time 712619306113 state 0x0 ref 1 00000100:00100000:0.0:1789760782.507213:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712619566529 time 712619306113 state 0x1 ref 1 00000100:00100000:0.0:1789760782.507246:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712619566529 time 712631852244 state 0x0 ref 1 00000100:00100000:0.0:1789760782.507258:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503823616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.507313:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77930496, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252a8 00010000:00010000:0.0:1789760782.507322:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.507337:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.507341:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.507347:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.507359:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.519792:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986333 is committed 00002000:00010000:0.0:1789760782.519933:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77930496 -> 77934592 00002000:00010000:0.0:1789760782.519939:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152232 -> 152240 00010000:00010000:0.0:1789760782.519953:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519966:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519974:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519980:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.519985:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519992:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.520010:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.520016:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1047 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77930496->77934591] (req 77930496->77934591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.520114:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503823616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12851us (13078us total) trans 4294986333 rc 0/0 00000100:00100000:0.0:1789760782.520124:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19037 00000100:00100000:0.0:1789760782.520133:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712619566529 time 712631852244 state 0x2 ref 0 00000100:00100000:2.0:1789760782.527110:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.527176:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503823744 00000100:00100000:0.0:1789760782.527186:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503823744 00000100:00100000:0.0:1789760782.527241:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712619566529 time 712631852244 state 0x0 ref 1 00000100:00100000:0.0:1789760782.527256:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712632112660 time 712631852244 state 0x1 ref 1 00000100:00100000:0.0:1789760782.527293:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712632112660 time 712651890281 state 0x0 ref 1 00000100:00100000:0.0:1789760782.527306:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503823744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.527360:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77934592, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252b0 00010000:00010000:0.0:1789760782.527370:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.527405:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.527411:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.527418:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.527431:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.534541:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986334 is committed 00002000:00010000:0.0:1789760782.534673:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77934592 -> 77938688 00002000:00010000:0.0:1789760782.534678:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152240 -> 152248 00010000:00010000:0.0:1789760782.534691:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.534700:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.534705:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.534711:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.534715:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.534724:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.534746:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.534758:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f104e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77934592->77938687] (req 77934592->77938687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.534862:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503823744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7553us (7757us total) trans 4294986334 rc 0/0 00000100:00100000:0.0:1789760782.534872:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19038 00000100:00100000:0.0:1789760782.534882:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712632112660 time 712651890281 state 0x2 ref 0 00000100:00100000:2.0:1789760782.540727:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.541065:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503823872 00000100:00100000:1.0:1789760782.541141:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503823872 00000100:00100000:1.0:1789760782.541196:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712632112660 time 712651890281 state 0x0 ref 1 00000100:00100000:1.0:1789760782.541210:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712652150697 time 712651890281 state 0x1 ref 1 00000100:00100000:1.0:1789760782.541241:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712652150697 time 712665847783 state 0x0 ref 1 00000100:00100000:1.0:1789760782.541253:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503823872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.541300:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77938688, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252b8 00010000:00010000:1.0:1789760782.541308:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.541324:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.541327:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.541334:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.541347:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.547887:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986335 is committed 00002000:00010000:1.0:1789760782.548001:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77938688 -> 77942784 00002000:00010000:1.0:1789760782.548007:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152248 -> 152256 00010000:00010000:1.0:1789760782.548020:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.548032:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.548042:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.548054:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.548059:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.548068:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.548086:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.548093:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1055 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77938688->77942783] (req 77938688->77942783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.548190:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503823872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6935us (7469us total) trans 4294986335 rc 0/0 00000100:00100000:1.0:1789760782.548203:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19039 00000100:00100000:1.0:1789760782.548214:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712652150697 time 712665847783 state 0x2 ref 0 00000100:00100000:2.0:1789760782.555228:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.555298:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824000 00000100:00100000:0.0:1789760782.555308:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824000 00000100:00100000:0.0:1789760782.555373:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712652150697 time 712665847783 state 0x0 ref 1 00000100:00100000:0.0:1789760782.555428:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712666108199 time 712665847783 state 0x1 ref 1 00000100:00100000:0.0:1789760782.555936:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712666108199 time 712680540239 state 0x0 ref 1 00000100:00100000:1.0:1789760782.556123:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503824000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.556289:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77942784, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252c0 00010000:00010000:1.0:1789760782.556300:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.556316:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.556331:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.556338:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.556354:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.563338:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986336 is committed 00002000:00010000:1.0:1789760782.563695:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77942784 -> 77946880 00002000:00010000:1.0:1789760782.563700:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152256 -> 152264 00010000:00010000:1.0:1789760782.563713:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.563725:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.563734:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.563741:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.563746:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.563755:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.563775:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.563788:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f105c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77942784->77946879] (req 77942784->77946879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.563932:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503824000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7978us (8714us total) trans 4294986336 rc 0/0 00000100:00100000:1.0:1789760782.563952:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19040 00000100:00100000:1.0:1789760782.563963:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712666108199 time 712680540239 state 0x2 ref 0 00000100:00100000:2.0:1789760782.566689:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.566750:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824128 00000100:00100000:0.0:1789760782.566759:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824128 00000100:00100000:0.0:1789760782.566817:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712666108199 time 712680540239 state 0x0 ref 1 00000100:00100000:0.0:1789760782.566833:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712680800655 time 712680540239 state 0x1 ref 1 00000100:00100000:0.0:1789760782.566858:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712680800655 time 712691464125 state 0x0 ref 1 00000100:00100000:0.0:1789760782.566870:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.566917:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77946880, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252c8 00010000:00010000:0.0:1789760782.566927:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.566951:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.566957:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.566963:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.566979:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.573058:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986337 is committed 00002000:00010000:0.0:1789760782.573168:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77946880 -> 77950976 00002000:00010000:0.0:1789760782.573176:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152264 -> 152272 00010000:00010000:0.0:1789760782.573190:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573203:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573212:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573220:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.573225:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573235:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573257:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573264:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1063 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77946880->77950975] (req 77946880->77950975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.573519:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6646us (6835us total) trans 4294986337 rc 0/0 00000100:00100000:0.0:1789760782.573530:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19041 00000100:00100000:0.0:1789760782.573540:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712680800655 time 712691464125 state 0x2 ref 0 00000100:00100000:2.0:1789760782.578375:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.578517:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824256 00000100:00100000:0.0:1789760782.578528:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824256 00000100:00100000:0.0:1789760782.578593:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712680800655 time 712691464125 state 0x0 ref 1 00000100:00100000:0.0:1789760782.578618:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712691724541 time 712691464125 state 0x1 ref 1 00000100:00100000:0.0:1789760782.578640:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712691724541 time 712703247008 state 0x0 ref 1 00000100:00100000:0.0:1789760782.578652:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503824256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.578698:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77950976, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252d0 00010000:00010000:0.0:1789760782.578708:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.578723:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.578728:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.578735:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.578749:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.584828:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986338 is committed 00002000:00010000:0.0:1789760782.585036:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77950976 -> 77955072 00002000:00010000:0.0:1789760782.585041:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152272 -> 152280 00010000:00010000:0.0:1789760782.585054:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585067:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585076:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585083:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.585088:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585098:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585117:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585130:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f106a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77950976->77955071] (req 77950976->77955071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.585224:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503824256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6569us (6864us total) trans 4294986338 rc 0/0 00000100:00100000:0.0:1789760782.585234:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19042 00000100:00100000:0.0:1789760782.585244:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712691724541 time 712703247008 state 0x2 ref 0 00000100:00100000:2.0:1789760782.590773:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.590867:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824384 00000100:00100000:1.0:1789760782.590877:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824384 00000100:00100000:1.0:1789760782.590941:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712691724541 time 712703247008 state 0x0 ref 1 00000100:00100000:1.0:1789760782.590956:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712703507424 time 712703247008 state 0x1 ref 1 00000100:00100000:1.0:1789760782.590981:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712703507424 time 712715588359 state 0x0 ref 1 00000100:00100000:1.0:1789760782.590994:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503824384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.591039:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77955072, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252d8 00010000:00010000:1.0:1789760782.591049:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.591064:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.591068:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.591074:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.591092:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.595526:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986339 is committed 00002000:00010000:2.0:1789760782.595791:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77955072 -> 77959168 00002000:00010000:2.0:1789760782.595797:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152280 -> 152288 00010000:00010000:2.0:1789760782.595811:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.595824:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.595834:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.596464:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.596470:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.596482:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.596501:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.596514:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1071 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77955072->77959167] (req 77955072->77959167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.596697:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503824384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5701us (5928us total) trans 4294986339 rc 0/0 00000100:00100000:2.0:1789760782.596708:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19043 00000100:00100000:2.0:1789760782.596717:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712703507424 time 712715588359 state 0x2 ref 0 00000100:00100000:2.0:1789760782.601792:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.602004:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824512 00000100:00100000:3.0:1789760782.602015:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824512 00000100:00100000:3.0:1789760782.602068:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712703507424 time 712715588359 state 0x0 ref 1 00000100:00100000:3.0:1789760782.602083:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712715848775 time 712715588359 state 0x1 ref 1 00000100:00100000:3.0:1789760782.602148:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712715848775 time 712726753622 state 0x0 ref 1 00000100:00100000:3.0:1789760782.602161:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.602207:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77959168, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252e0 00010000:00010000:3.0:1789760782.602217:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.602231:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760782.602235:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.602242:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.602259:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.606699:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986340 is committed 00002000:00010000:2.0:1789760782.607498:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77959168 -> 77963264 00002000:00010000:2.0:1789760782.607505:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152288 -> 152296 00010000:00010000:2.0:1789760782.607518:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.607530:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.607540:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.607547:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.607552:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.607569:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.607590:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.607598:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1078 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77959168->77963263] (req 77959168->77963263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.607706:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5541us (5918us total) trans 4294986340 rc 0/0 00000100:00100000:2.0:1789760782.607715:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19044 00000100:00100000:2.0:1789760782.607725:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712715848775 time 712726753622 state 0x2 ref 0 00000100:00100000:1.0:1789760782.611817:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.613019:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824640 00000100:00100000:0.0:1789760782.613032:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824640 00000100:00100000:0.0:1789760782.613095:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712715848775 time 712726753622 state 0x0 ref 1 00000100:00100000:0.0:1789760782.613110:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712727014038 time 712726753622 state 0x1 ref 1 00000100:00100000:0.0:1789760782.613137:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712727014038 time 712737744247 state 0x0 ref 1 00000100:00100000:0.0:1789760782.613150:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.613199:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77963264, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252e8 00010000:00010000:0.0:1789760782.613209:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.613224:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.613229:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.613235:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.613252:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.617448:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986341 is committed 00002000:00010000:3.0:1789760782.617620:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77963264 -> 77967360 00002000:00010000:3.0:1789760782.617626:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152296 -> 152304 00010000:00010000:3.0:1789760782.617641:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.617655:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.617664:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.617672:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.617677:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.617686:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.617706:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.617714:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f107f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77963264->77967359] (req 77963264->77967359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.617926:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4772us (6113us total) trans 4294986341 rc 0/0 00000100:00100000:3.0:1789760782.617939:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19045 00000100:00100000:3.0:1789760782.617949:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712727014038 time 712737744247 state 0x2 ref 0 00000100:00100000:3.0:1789760782.622288:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.622575:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824768 00000100:00100000:0.0:1789760782.622584:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824768 00000100:00100000:0.0:1789760782.622652:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712727014038 time 712737744247 state 0x0 ref 1 00000100:00100000:0.0:1789760782.622669:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712738004663 time 712737744247 state 0x1 ref 1 00000100:00100000:0.0:1789760782.622701:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712738004663 time 712747306198 state 0x0 ref 1 00000100:00100000:0.0:1789760782.622713:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503824768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.622761:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77967360, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252f0 00010000:00010000:0.0:1789760782.622771:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.622788:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.622792:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.622798:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.622812:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.628972:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986342 is committed 00002000:00010000:0.0:1789760782.629086:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77967360 -> 77971456 00002000:00010000:0.0:1789760782.629093:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152304 -> 152312 00010000:00010000:0.0:1789760782.629107:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.629131:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.629140:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.629147:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.629152:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.629163:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.629184:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.629193:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1086 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77967360->77971455] (req 77967360->77971455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.629284:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503824768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6568us (7003us total) trans 4294986342 rc 0/0 00000100:00100000:0.0:1789760782.629295:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19046 00000100:00100000:0.0:1789760782.629305:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712738004663 time 712747306198 state 0x2 ref 0 00000100:00100000:2.0:1789760782.634450:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.634582:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503824896 00000100:00100000:1.0:1789760782.634592:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503824896 00000100:00100000:1.0:1789760782.634658:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712738004663 time 712747306198 state 0x0 ref 1 00000100:00100000:1.0:1789760782.634673:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712747566614 time 712747306198 state 0x1 ref 1 00000100:00100000:1.0:1789760782.634704:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712747566614 time 712759310682 state 0x0 ref 1 00000100:00100000:1.0:1789760782.634717:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc15050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.634768:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77971456, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x252f8 00010000:00010000:1.0:1789760782.634777:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.634793:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.634798:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.634805:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.634819:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.640055:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986343 is committed 00002000:00010000:1.0:1789760782.640271:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77971456 -> 77975552 00002000:00010000:1.0:1789760782.640276:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152312 -> 152320 00010000:00010000:1.0:1789760782.640298:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.640312:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.640321:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.640329:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.640334:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.640344:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.640363:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.640371:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f108d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77971456->77975551] (req 77971456->77975551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.640886:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc15050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503824896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6166us (6440us total) trans 4294986343 rc 0/0 00000100:00100000:1.0:1789760782.640898:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19047 00000100:00100000:1.0:1789760782.640908:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712747566614 time 712759310682 state 0x2 ref 0 00000100:00100000:1.0:1789760782.660781:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.661069:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825024 00000100:00100000:2.0:1789760782.661079:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825024 00000100:00100000:2.0:1789760782.661134:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712747566614 time 712759310682 state 0x0 ref 1 00000100:00100000:2.0:1789760782.661149:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712759571098 time 712759310682 state 0x1 ref 1 00000100:00100000:2.0:1789760782.661183:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712759571098 time 712785789352 state 0x0 ref 1 00000100:00100000:2.0:1789760782.661196:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.661484:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77975552, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25300 00010000:00010000:2.0:1789760782.661495:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.661509:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.661515:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.661521:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.661535:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.672428:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986344 is committed 00002000:00010000:2.0:1789760782.673192:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77975552 -> 77979648 00002000:00010000:2.0:1789760782.673198:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152320 -> 152328 00010000:00010000:2.0:1789760782.673211:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.673223:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.673303:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.673311:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.673316:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.673327:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.673349:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.673357:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1094 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77975552->77979647] (req 77975552->77979647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.673507:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12308us (12913us total) trans 4294986344 rc 0/0 00000100:00100000:2.0:1789760782.673518:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19048 00000100:00100000:2.0:1789760782.673527:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712759571098 time 712785789352 state 0x2 ref 0 00000100:00100000:3.0:1789760782.677327:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.677609:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825152 00000100:00100000:1.0:1789760782.677619:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825152 00000100:00100000:1.0:1789760782.677673:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712759571098 time 712785789352 state 0x0 ref 1 00000100:00100000:1.0:1789760782.677689:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712786049768 time 712785789352 state 0x1 ref 1 00000100:00100000:1.0:1789760782.677719:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712786049768 time 712802326051 state 0x0 ref 1 00000100:00100000:1.0:1789760782.677730:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503825152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.677775:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77979648, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25308 00010000:00010000:1.0:1789760782.677784:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.677798:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.677803:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.677809:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.677822:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.685942:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986345 is committed 00002000:00010000:1.0:1789760782.686119:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77979648 -> 77983744 00002000:00010000:1.0:1789760782.686124:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152328 -> 152336 00010000:00010000:1.0:1789760782.686138:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.686150:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.686159:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.686166:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.686171:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.686181:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.686201:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.686209:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f109b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77979648->77983743] (req 77979648->77983743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.686315:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503825152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8581us (8992us total) trans 4294986345 rc 0/0 00000100:00100000:1.0:1789760782.686326:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19049 00000100:00100000:1.0:1789760782.686336:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712786049768 time 712802326051 state 0x2 ref 0 00000100:00100000:3.0:1789760782.689872:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.689983:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825280 00000100:00100000:0.0:1789760782.689994:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825280 00000100:00100000:0.0:1789760782.690049:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712786049768 time 712802326051 state 0x0 ref 1 00000100:00100000:0.0:1789760782.690064:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712802586467 time 712802326051 state 0x1 ref 1 00000100:00100000:0.0:1789760782.690095:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712802586467 time 712814701177 state 0x0 ref 1 00000100:00100000:0.0:1789760782.690107:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.690156:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77983744, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25310 00010000:00010000:0.0:1789760782.690165:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.690180:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.690184:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.690190:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.690202:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.695862:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986346 is committed 00002000:00010000:0.0:1789760782.696480:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77983744 -> 77987840 00002000:00010000:0.0:1789760782.696487:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152336 -> 152344 00010000:00010000:0.0:1789760782.696501:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696519:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696527:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696534:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.696539:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696547:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696619:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696628:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77983744->77987839] (req 77983744->77987839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.697621:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7507us (7750us total) trans 4294986346 rc 0/0 00000100:00100000:0.0:1789760782.697635:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19050 00000100:00100000:0.0:1789760782.697646:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712802586467 time 712814701177 state 0x2 ref 0 00000100:00100000:2.0:1789760782.703696:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.703923:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825408 00000100:00100000:3.0:1789760782.703933:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825408 00000100:00100000:3.0:1789760782.703990:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712802586467 time 712814701177 state 0x0 ref 1 00000100:00100000:3.0:1789760782.704007:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712814961593 time 712814701177 state 0x1 ref 1 00000100:00100000:3.0:1789760782.704039:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712814961593 time 712828645342 state 0x0 ref 1 00000100:00100000:3.0:1789760782.704051:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503825408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.704098:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77987840, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25318 00010000:00010000:3.0:1789760782.704107:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.704121:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760782.704126:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.704132:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.704150:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.715024:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986347 is committed 00002000:00010000:3.0:1789760782.715451:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77987840 -> 77991936 00002000:00010000:3.0:1789760782.715455:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152344 -> 152352 00010000:00010000:3.0:1789760782.715466:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.715476:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.715481:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.715486:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.715489:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.715496:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.715510:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.715514:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77987840->77991935] (req 77987840->77991935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.715802:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503825408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11745us (12197us total) trans 4294986347 rc 0/0 00000100:00100000:2.0:1789760782.715814:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19051 00000100:00100000:2.0:1789760782.715825:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712814961593 time 712828645342 state 0x2 ref 0 00000100:00100000:3.0:1789760782.719057:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.719615:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825536 00000100:00100000:2.0:1789760782.719626:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825536 00000100:00100000:2.0:1789760782.719680:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712814961593 time 712828645342 state 0x0 ref 1 00000100:00100000:2.0:1789760782.719696:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712828905758 time 712828645342 state 0x1 ref 1 00000100:00100000:2.0:1789760782.719733:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712828905758 time 712844338895 state 0x0 ref 1 00000100:00100000:2.0:1789760782.719749:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.719799:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77991936, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25320 00010000:00010000:2.0:1789760782.719808:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.719824:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.719828:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.719834:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.719849:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.726473:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986348 is committed 00002000:00010000:2.0:1789760782.726653:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77991936 -> 77996032 00002000:00010000:2.0:1789760782.726658:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152352 -> 152360 00010000:00010000:2.0:1789760782.726671:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.726684:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.726693:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.726700:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.726706:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.726714:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.726733:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.726741:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f10b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77991936->77996031] (req 77991936->77996031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.726840:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7087us (7785us total) trans 4294986348 rc 0/0 00000100:00100000:2.0:1789760782.726850:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19052 00000100:00100000:2.0:1789760782.726859:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712828905758 time 712844338895 state 0x2 ref 0 00000100:00100000:3.0:1789760782.730476:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.731521:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825664 00000100:00100000:1.0:1789760782.731533:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825664 00000100:00100000:1.0:1789760782.731598:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712828905758 time 712844338895 state 0x0 ref 1 00000100:00100000:1.0:1789760782.731612:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712844599311 time 712844338895 state 0x1 ref 1 00000100:00100000:2.0:1789760782.731794:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712844599311 time 712856397345 state 0x0 ref 1 00000100:00100000:2.0:1789760782.731811:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503825664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760782.731861:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 77996032, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25328 00010000:00010000:2.0:1789760782.731871:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.731885:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760782.731889:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.731896:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.731910:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.739149:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986349 is committed 00002000:00010000:1.0:1789760782.739261:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 77996032 -> 78000128 00002000:00010000:1.0:1789760782.739267:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152360 -> 152368 00010000:00010000:1.0:1789760782.739280:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.739304:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.739313:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.739321:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.739326:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.739335:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.739359:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.739371:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f10b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [77996032->78000127] (req 77996032->78000127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.739503:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503825664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7691us (9031us total) trans 4294986349 rc 0/0 00000100:00100000:1.0:1789760782.739514:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19053 00000100:00100000:1.0:1789760782.739523:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712844599311 time 712856397345 state 0x2 ref 0 00000100:00100000:1.0:1789760782.743016:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.743243:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825792 00000100:00100000:0.0:1789760782.743253:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825792 00000100:00100000:0.0:1789760782.743321:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712844599311 time 712856397345 state 0x0 ref 1 00000100:00100000:0.0:1789760782.743337:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712856657761 time 712856397345 state 0x1 ref 1 00000100:00100000:0.0:1789760782.743367:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712856657761 time 712867973329 state 0x0 ref 1 00000100:00100000:0.0:1789760782.743379:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.743445:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78000128, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25330 00010000:00010000:0.0:1789760782.743454:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743470:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760782.743475:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.743481:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743493:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.750901:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986350 is committed 00002000:00010000:3.0:1789760782.751054:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78000128 -> 78004224 00002000:00010000:3.0:1789760782.751059:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152368 -> 152376 00010000:00010000:3.0:1789760782.751072:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.751084:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.751094:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.751102:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.751107:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.751117:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.751137:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.751144:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f10be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78000128->78004223] (req 78000128->78004223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.751246:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7863us (8234us total) trans 4294986350 rc 0/0 00000100:00100000:3.0:1789760782.751257:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19054 00000100:00100000:3.0:1789760782.751277:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712856657761 time 712867973329 state 0x2 ref 0 00000100:00100000:3.0:1789760782.756534:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.756986:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503825920 00000100:00100000:1.0:1789760782.756996:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503825920 00000100:00100000:1.0:1789760782.757049:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712856657761 time 712867973329 state 0x0 ref 1 00000100:00100000:1.0:1789760782.757063:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712868233745 time 712867973329 state 0x1 ref 1 00000100:00100000:1.0:1789760782.757096:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712868233745 time 712881702788 state 0x0 ref 1 00000100:00100000:1.0:1789760782.757108:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.757150:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78004224, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25338 00010000:00010000:1.0:1789760782.757158:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.757177:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.757182:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.757187:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.757200:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.764213:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986351 is committed 00002000:00010000:1.0:1789760782.764829:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78004224 -> 78008320 00002000:00010000:1.0:1789760782.764835:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152376 -> 152384 00010000:00010000:1.0:1789760782.764847:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.764869:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.764877:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.764884:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.764888:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.764905:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.764928:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.764935:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78004224->78008319] (req 78004224->78008319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.765040:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503825920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7929us (8511us total) trans 4294986351 rc 0/0 00000100:00100000:1.0:1789760782.765050:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19055 00000100:00100000:1.0:1789760782.765060:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712868233745 time 712881702788 state 0x2 ref 0 00000100:00100000:2.0:1789760782.772802:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.772944:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826048 00000100:00100000:1.0:1789760782.772956:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826048 00000100:00100000:1.0:1789760782.773011:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712868233745 time 712881702788 state 0x0 ref 1 00000100:00100000:1.0:1789760782.773025:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712881963204 time 712881702788 state 0x1 ref 1 00000100:00100000:1.0:1789760782.773055:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712881963204 time 712897662119 state 0x0 ref 1 00000100:00100000:1.0:1789760782.773068:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503826048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.773113:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78008320, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25340 00010000:00010000:1.0:1789760782.773121:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.773136:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.773140:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.773147:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.773160:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.780021:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986352 is committed 00002000:00010000:1.0:1789760782.780958:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78008320 -> 78012416 00002000:00010000:1.0:1789760782.780965:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152384 -> 152392 00010000:00010000:1.0:1789760782.780978:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.780990:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.780999:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.781007:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.781012:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.781022:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.781040:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.781052:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78008320->78012415] (req 78008320->78012415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.781164:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503826048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8093us (8365us total) trans 4294986352 rc 0/0 00000100:00100000:1.0:1789760782.781174:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19056 00000100:00100000:1.0:1789760782.781184:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712881963204 time 712897662119 state 0x2 ref 0 00000100:00100000:2.0:1789760782.787994:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.788133:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826176 00000100:00100000:1.0:1789760782.788143:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826176 00000100:00100000:1.0:1789760782.788196:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712881963204 time 712897662119 state 0x0 ref 1 00000100:00100000:1.0:1789760782.788211:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712897922535 time 712897662119 state 0x1 ref 1 00000100:00100000:1.0:1789760782.788242:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712897922535 time 712912848235 state 0x0 ref 1 00000100:00100000:1.0:1789760782.788256:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503826176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.788314:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78012416, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x25348 00010000:00010000:1.0:1789760782.788322:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.788337:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.788341:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.788347:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.788367:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.789332:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.789369:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.789432:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.789460:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.789480:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760782.789501:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826432 00000100:00100000:2.0:1789760782.789507:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826432 00000100:00100000:2.0:1789760782.789549:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826688 00000100:00100000:2.0:1789760782.789552:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826688 00000100:00100000:2.0:1789760782.789575:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826560 00000100:00100000:2.0:1789760782.789578:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826560 00000100:00100000:2.0:1789760782.789589:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826816 00000100:00100000:2.0:1789760782.789591:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826816 00000100:00100000:2.0:1789760782.789728:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 971 00000100:00100000:2.0:1789760782.789736:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503826432:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760782.789789:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503826432:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 55us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.789794:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 971 00000100:00100000:2.0:1789760782.789806:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 972 00000100:00100000:2.0:1789760782.789810:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503826688:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760782.789823:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503826688:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 13us (455us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.789827:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 972 00000100:00100000:2.0:1789760782.789833:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 973 00000100:00100000:2.0:1789760782.789836:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503826560:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760782.789847:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503826560:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 11us (388us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.789850:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 973 00000100:00100000:2.0:1789760782.789857:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 974 00000100:00100000:2.0:1789760782.789863:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503826816:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760782.789883:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503826816:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 20us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.789888:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 974 00000100:00080000:2.0:1789760782.789906:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760782.789909:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760782.789967:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826304 00000100:00100000:0.0:1789760782.789971:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826304 00000100:00100000:0.0:1789760782.790006:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 490 00000100:00100000:0.0:1789760782.790013:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503826304:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760782.790040:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503826304:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 28us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.790047:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 490 00000001:00080000:0.0:1789760782.793933:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986353 is committed 00002000:00010000:1.0:1789760782.794201:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78012416 -> 78016512 00002000:00010000:1.0:1789760782.794204:0:8434:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:1.0:1789760782.794206:0:8434:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00002000:00010000:1.0:1789760782.794208:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152392 -> 152400 00010000:00010000:1.0:1789760782.794219:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.794227:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.794233:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.794238:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.794241:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.794247:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.794260:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.794269:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78012416->78016511] (req 78012416->78016511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.794353:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503826176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6095us (6364us total) trans 4294986353 rc 0/0 00000100:00100000:1.0:1789760782.794363:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19057 00000100:00100000:1.0:1789760782.794372:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712897922535 time 712912848235 state 0x2 ref 0 00000100:00100000:2.0:1789760782.799281:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.799966:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503826944 00000100:00100000:3.0:1789760782.799974:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503826944 00000100:00100000:3.0:1789760782.800023:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712897922535 time 712912848235 state 0x0 ref 1 00000100:00100000:3.0:1789760782.800034:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712913108651 time 712912848235 state 0x1 ref 1 00000100:00100000:3.0:1789760782.800057:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712913108651 time 712924664742 state 0x0 ref 1 00000100:00100000:3.0:1789760782.800069:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503826944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.800126:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78016512, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25350 00010000:00010000:3.0:1789760782.800135:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.800150:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760782.800154:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.800161:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.800175:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.805731:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986354 is committed 00002000:00010000:3.0:1789760782.806102:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78016512 -> 78020608 00002000:00010000:3.0:1789760782.806107:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152400 -> 152408 00010000:00010000:3.0:1789760782.806122:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.806135:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.806144:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.806152:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.806157:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.806166:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.806186:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.806194:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78016512->78020607] (req 78016512->78020607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.806896:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503826944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6319us (7117us total) trans 4294986354 rc 0/0 00000100:00100000:3.0:1789760782.806909:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19058 00000100:00100000:3.0:1789760782.806919:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712913108651 time 712924664742 state 0x2 ref 0 00000100:00100000:2.0:1789760782.809721:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.809934:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827072 00000100:00100000:1.0:1789760782.809944:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827072 00000100:00100000:1.0:1789760782.809998:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712913108651 time 712924664742 state 0x0 ref 1 00000100:00100000:1.0:1789760782.810013:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712924925158 time 712924664742 state 0x1 ref 1 00000100:00100000:1.0:1789760782.810044:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712924925158 time 712934650380 state 0x0 ref 1 00000100:00100000:1.0:1789760782.810055:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503827072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.810117:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78020608, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25358 00010000:00010000:1.0:1789760782.810126:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.810142:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.810146:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.810152:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.810172:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.817543:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986355 is committed 00002000:00010000:1.0:1789760782.817682:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78020608 -> 78024704 00002000:00010000:1.0:1789760782.817687:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152408 -> 152416 00010000:00010000:1.0:1789760782.817700:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.817714:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.817723:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.817730:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.817735:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.817745:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.817776:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.817783:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f10e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78020608->78024703] (req 78020608->78024703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.817980:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503827072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7922us (8262us total) trans 4294986355 rc 0/0 00000100:00100000:1.0:1789760782.817992:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19059 00000100:00100000:1.0:1789760782.818001:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712924925158 time 712934650380 state 0x2 ref 0 00000100:00100000:0.0:1789760782.822690:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.822918:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827200 00000100:00100000:1.0:1789760782.822930:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827200 00000100:00100000:1.0:1789760782.822988:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712924925158 time 712934650380 state 0x0 ref 1 00000100:00100000:1.0:1789760782.823004:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712934910796 time 712934650380 state 0x1 ref 1 00000100:00100000:1.0:1789760782.823036:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712934910796 time 712947642305 state 0x0 ref 1 00000100:00100000:1.0:1789760782.823048:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503827200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.823094:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78024704, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25360 00010000:00010000:1.0:1789760782.823102:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.823118:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.823123:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.823130:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.823143:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.829378:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986356 is committed 00002000:00010000:2.0:1789760782.829722:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78024704 -> 78028800 00002000:00010000:2.0:1789760782.829728:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152416 -> 152424 00010000:00010000:2.0:1789760782.829743:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.829755:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.829769:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.829777:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.829782:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.829791:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.829812:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.829820:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f10e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78024704->78028799] (req 78024704->78028799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.829914:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503827200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6863us (7229us total) trans 4294986356 rc 0/0 00000100:00100000:2.0:1789760782.829925:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19060 00000100:00100000:2.0:1789760782.829935:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712934910796 time 712947642305 state 0x2 ref 0 00000100:00100000:2.0:1789760782.836677:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.836853:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827328 00000100:00100000:1.0:1789760782.836862:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827328 00000100:00100000:1.0:1789760782.836919:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712934910796 time 712947642305 state 0x0 ref 1 00000100:00100000:1.0:1789760782.836934:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712947902721 time 712947642305 state 0x1 ref 1 00000100:00100000:1.0:1789760782.837010:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712947902721 time 712961616009 state 0x0 ref 1 00000100:00100000:1.0:1789760782.837025:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503827328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.837073:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78028800, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25368 00010000:00010000:1.0:1789760782.837083:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.837097:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.837102:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.837108:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.837130:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.844860:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986357 is committed 00002000:00010000:1.0:1789760782.844900:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78028800 -> 78032896 00002000:00010000:1.0:1789760782.844971:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152424 -> 152432 00010000:00010000:1.0:1789760782.844987:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845000:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845009:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845016:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.845020:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845029:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845048:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845056:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78028800->78032895] (req 78028800->78032895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.845146:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503827328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8120us (8473us total) trans 4294986357 rc 0/0 00000100:00100000:1.0:1789760782.845157:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19061 00000100:00100000:1.0:1789760782.845166:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712947902721 time 712961616009 state 0x2 ref 0 00000100:00100000:2.0:1789760782.848586:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.848892:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827456 00000100:00100000:3.0:1789760782.848901:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827456 00000100:00100000:3.0:1789760782.848956:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712947902721 time 712961616009 state 0x0 ref 1 00000100:00100000:3.0:1789760782.848970:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712961876425 time 712961616009 state 0x1 ref 1 00000100:00100000:3.0:1789760782.849002:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712961876425 time 712973608480 state 0x0 ref 1 00000100:00100000:3.0:1789760782.849014:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503827456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.849057:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78032896, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25370 00010000:00010000:3.0:1789760782.849067:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.849084:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760782.849089:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.849095:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.849109:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.855643:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986358 is committed 00002000:00010000:3.0:1789760782.857264:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78032896 -> 78036992 00002000:00010000:3.0:1789760782.857277:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152432 -> 152440 00010000:00010000:3.0:1789760782.857294:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.857308:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.857316:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.857324:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.857329:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.857339:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.857370:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.857379:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f10f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78032896->78036991] (req 78032896->78036991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.857610:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503827456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8584us (9019us total) trans 4294986358 rc 0/0 00000100:00100000:3.0:1789760782.857622:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19062 00000100:00100000:3.0:1789760782.857631:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712961876425 time 712973608480 state 0x2 ref 0 00000100:00100000:0.0:1789760782.863785:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.863901:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827584 00000100:00100000:1.0:1789760782.863911:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827584 00000100:00100000:1.0:1789760782.863967:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712961876425 time 712973608480 state 0x0 ref 1 00000100:00100000:1.0:1789760782.863982:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712973868896 time 712973608480 state 0x1 ref 1 00000100:00100000:1.0:1789760782.864016:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712973868896 time 712988622635 state 0x0 ref 1 00000100:00100000:1.0:1789760782.864029:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503827584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.864075:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78036992, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25378 00010000:00010000:1.0:1789760782.864086:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.864102:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.864106:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.864113:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.864127:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.866888:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760782.869515:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986359 is committed 00002000:00010000:0.0:1789760782.869645:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78036992 -> 78041088 00002000:00010000:0.0:1789760782.869651:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152440 -> 152448 00010000:00010000:0.0:1789760782.869665:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.869679:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.869688:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.869696:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.869700:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.869710:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.869730:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.869738:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f10fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78036992->78041087] (req 78036992->78041087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.869830:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503827584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5797us (6049us total) trans 4294986359 rc 0/0 00000100:00100000:0.0:1789760782.869841:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19063 00000100:00100000:0.0:1789760782.869852:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712973868896 time 712988622635 state 0x2 ref 0 00000100:00100000:2.0:1789760782.875189:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.875257:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827712 00000100:00100000:1.0:1789760782.875267:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827712 00000100:00100000:1.0:1789760782.875322:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712973868896 time 712988622635 state 0x0 ref 1 00000100:00100000:1.0:1789760782.875338:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712988883051 time 712988622635 state 0x1 ref 1 00000100:00100000:1.0:1789760782.875376:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712988883051 time 712999982362 state 0x0 ref 1 00000100:00100000:1.0:1789760782.875415:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503827712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.875463:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78041088, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25380 00010000:00010000:1.0:1789760782.875472:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.875487:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760782.875492:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.875498:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.875512:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.880263:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986360 is committed 00002000:00010000:1.0:1789760782.880708:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78041088 -> 78045184 00002000:00010000:1.0:1789760782.880714:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152448 -> 152456 00010000:00010000:1.0:1789760782.880726:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.880737:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.880747:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.880754:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.880759:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.880768:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.880788:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.880795:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1104 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78041088->78045183] (req 78041088->78045183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.880936:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503827712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5542us (5757us total) trans 4294986360 rc 0/0 00000100:00100000:1.0:1789760782.880946:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19064 00000100:00100000:1.0:1789760782.880955:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712988883051 time 712999982362 state 0x2 ref 0 00000100:00100000:2.0:1789760782.888020:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.888187:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827840 00000100:00100000:1.0:1789760782.888197:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827840 00000100:00100000:1.0:1789760782.888248:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 712988883051 time 712999982362 state 0x0 ref 1 00000100:00100000:1.0:1789760782.888264:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713000242778 time 712999982362 state 0x1 ref 1 00000100:00100000:1.0:1789760782.888296:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713000242778 time 713012902399 state 0x0 ref 1 00000100:00100000:1.0:1789760782.888309:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503827840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.888355:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78045184, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25388 00010000:00010000:1.0:1789760782.888364:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.888380:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760782.888385:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.888426:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.888442:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.895041:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986361 is committed 00002000:00010000:1.0:1789760782.895228:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78045184 -> 78049280 00002000:00010000:1.0:1789760782.895234:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152456 -> 152464 00010000:00010000:1.0:1789760782.895246:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.895258:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.895268:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.895276:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.896752:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.896773:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.896795:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.896804:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f110b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78045184->78049279] (req 78045184->78049279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.896904:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503827840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8593us (8890us total) trans 4294986361 rc 0/0 00000100:00100000:1.0:1789760782.896922:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19065 00000100:00100000:1.0:1789760782.896932:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713000242778 time 713012902399 state 0x2 ref 0 00000100:00100000:0.0:1789760782.900599:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.900725:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503827968 00000100:00100000:3.0:1789760782.900734:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503827968 00000100:00100000:3.0:1789760782.900790:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713000242778 time 713012902399 state 0x0 ref 1 00000100:00100000:3.0:1789760782.900805:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713013162815 time 713012902399 state 0x1 ref 1 00000100:00100000:3.0:1789760782.900838:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713013162815 time 713025444774 state 0x0 ref 1 00000100:00100000:3.0:1789760782.900850:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503827968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.900896:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78049280, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25390 00010000:00010000:3.0:1789760782.900905:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.900920:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760782.900924:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.900930:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.900944:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.905737:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986362 is committed 00002000:00010000:3.0:1789760782.905997:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78049280 -> 78053376 00002000:00010000:3.0:1789760782.906004:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152464 -> 152472 00010000:00010000:3.0:1789760782.906021:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.906036:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.906047:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.906055:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.906060:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.906071:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.906091:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.906100:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1112 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78049280->78053375] (req 78049280->78053375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.906276:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503827968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5423us (5686us total) trans 4294986362 rc 0/0 00000100:00100000:3.0:1789760782.906636:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19066 00000100:00100000:3.0:1789760782.906657:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713013162815 time 713025444774 state 0x2 ref 0 00000100:00100000:2.0:1789760782.910503:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.910579:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828096 00000100:00100000:0.0:1789760782.910589:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828096 00000100:00100000:0.0:1789760782.910762:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713013162815 time 713025444774 state 0x0 ref 1 00000100:00100000:0.0:1789760782.910777:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713025705190 time 713025444774 state 0x1 ref 1 00000100:00100000:0.0:1789760782.910799:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713025705190 time 713035406262 state 0x0 ref 1 00000100:00100000:0.0:1789760782.910811:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503828096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.910860:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78053376, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25398 00010000:00010000:0.0:1789760782.910870:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.910885:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760782.910889:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.910896:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.910909:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.918685:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986363 is committed 00002000:00010000:2.0:1789760782.918795:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78053376 -> 78057472 00002000:00010000:2.0:1789760782.918801:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152472 -> 152480 00010000:00010000:2.0:1789760782.918814:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.918826:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.918835:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.918842:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760782.918847:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.918856:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.918876:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.918884:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1119 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78053376->78057471] (req 78053376->78057471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.918982:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503828096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8169us (8485us total) trans 4294986363 rc 0/0 00000100:00100000:2.0:1789760782.918993:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19067 00000100:00100000:2.0:1789760782.919003:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713025705190 time 713035406262 state 0x2 ref 0 00000100:00100000:2.0:1789760782.922638:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.923845:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828224 00000100:00100000:1.0:1789760782.923858:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828224 00000100:00100000:1.0:1789760782.923920:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713025705190 time 713035406262 state 0x0 ref 1 00000100:00100000:1.0:1789760782.923937:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713035666678 time 713035406262 state 0x1 ref 1 00000100:00100000:1.0:1789760782.924006:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713035666678 time 713048611007 state 0x0 ref 1 00000100:00100000:1.0:1789760782.924019:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.924067:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78057472, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253a0 00010000:00010000:1.0:1789760782.924076:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.924093:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760782.924098:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.924104:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.924118:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.932044:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986364 is committed 00002000:00010000:1.0:1789760782.932168:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78057472 -> 78061568 00002000:00010000:1.0:1789760782.932172:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152480 -> 152488 00010000:00010000:1.0:1789760782.932185:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.932196:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.932205:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.932212:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.932216:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.932228:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.932247:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.932258:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1120 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78057472->78061567] (req 78057472->78061567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.932386:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8365us (9753us total) trans 4294986364 rc 0/0 00000100:00100000:1.0:1789760782.932438:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19068 00000100:00100000:1.0:1789760782.932448:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713035666678 time 713048611007 state 0x2 ref 0 00000100:00100000:3.0:1789760782.936580:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.936710:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828352 00000100:00100000:1.0:1789760782.936721:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828352 00000100:00100000:1.0:1789760782.936775:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713035666678 time 713048611007 state 0x0 ref 1 00000100:00100000:1.0:1789760782.936916:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713048871423 time 713048611007 state 0x1 ref 1 00000100:00100000:1.0:1789760782.937386:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713048871423 time 713061989825 state 0x0 ref 1 00000100:00100000:1.0:1789760782.937756:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503828352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.937817:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78061568, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253a8 00010000:00010000:1.0:1789760782.937826:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.937841:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760782.937845:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.937851:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.938341:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.944942:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986365 is committed 00002000:00010000:1.0:1789760782.945213:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78061568 -> 78065664 00002000:00010000:1.0:1789760782.945218:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152488 -> 152496 00010000:00010000:1.0:1789760782.945232:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.945245:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.945254:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.945261:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.945265:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.945274:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.945352:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.945361:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1127 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78061568->78065663] (req 78061568->78065663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.945588:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503828352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7832us (9027us total) trans 4294986365 rc 0/0 00000100:00100000:1.0:1789760782.945598:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19069 00000100:00100000:1.0:1789760782.945608:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713048871423 time 713061989825 state 0x2 ref 0 00000100:00100000:2.0:1789760782.949972:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.950041:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828480 00000100:00100000:3.0:1789760782.950049:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828480 00000100:00100000:3.0:1789760782.950154:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713048871423 time 713061989825 state 0x0 ref 1 00000100:00100000:3.0:1789760782.950168:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713062250241 time 713061989825 state 0x1 ref 1 00000100:00100000:3.0:1789760782.950222:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713062250241 time 713074828395 state 0x0 ref 1 00000100:00100000:3.0:1789760782.950234:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.950277:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78065664, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253b0 00010000:00010000:3.0:1789760782.950295:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.950309:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760782.950313:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.950320:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.950333:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.955592:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986366 is committed 00002000:00010000:3.0:1789760782.955704:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78065664 -> 78069760 00002000:00010000:3.0:1789760782.955709:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152496 -> 152504 00010000:00010000:3.0:1789760782.955723:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.955735:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.955744:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.955752:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760782.955758:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.955767:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.955787:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.955795:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f112e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78065664->78069759] (req 78065664->78069759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.955903:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5664us (5933us total) trans 4294986366 rc 0/0 00000100:00100000:3.0:1789760782.955915:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19070 00000100:00100000:3.0:1789760782.955924:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713062250241 time 713074828395 state 0x2 ref 0 00000100:00100000:2.0:1789760782.959364:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760782.959513:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828608 00000100:00100000:0.0:1789760782.959524:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828608 00000100:00100000:0.0:1789760782.959587:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713062250241 time 713074828395 state 0x0 ref 1 00000100:00100000:0.0:1789760782.959603:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713075088811 time 713074828395 state 0x1 ref 1 00000100:00100000:0.0:1789760782.959674:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713075088811 time 713084280260 state 0x0 ref 1 00000100:00100000:0.0:1789760782.959688:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760782.959741:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78069760, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253b8 00010000:00010000:0.0:1789760782.959750:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.959764:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760782.959769:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.959774:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.959784:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.965196:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986367 is committed 00002000:00010000:0.0:1789760782.965595:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78069760 -> 78073856 00002000:00010000:0.0:1789760782.965600:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152504 -> 152512 00010000:00010000:0.0:1789760782.965613:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.965633:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.965642:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.965650:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760782.965654:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.965664:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.965685:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.965694:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1135 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78069760->78073855] (req 78069760->78073855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.965790:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6099us (6429us total) trans 4294986367 rc 0/0 00000100:00100000:0.0:1789760782.965801:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19071 00000100:00100000:0.0:1789760782.965811:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713075088811 time 713084280260 state 0x2 ref 0 00000100:00100000:0.0:1789760782.972870:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.972973:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828736 00000100:00100000:1.0:1789760782.972983:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828736 00000100:00100000:1.0:1789760782.973046:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713075088811 time 713084280260 state 0x0 ref 1 00000100:00100000:1.0:1789760782.973060:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713084540676 time 713084280260 state 0x1 ref 1 00000100:00100000:1.0:1789760782.973085:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713084540676 time 713097692203 state 0x0 ref 1 00000100:00100000:1.0:1789760782.973097:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.973140:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78073856, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253c0 00010000:00010000:1.0:1789760782.973148:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.973163:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760782.973168:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.973173:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.973187:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.978771:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986368 is committed 00002000:00010000:1.0:1789760782.979143:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78073856 -> 78077952 00002000:00010000:1.0:1789760782.979149:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152512 -> 152520 00010000:00010000:1.0:1789760782.979162:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.979175:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.979184:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.979192:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.979197:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.979206:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.979226:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.979248:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f113c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78073856->78077951] (req 78073856->78077951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.980242:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7139us (7374us total) trans 4294986368 rc 0/0 00000100:00100000:1.0:1789760782.980254:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19072 00000100:00100000:1.0:1789760782.980264:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713084540676 time 713097692203 state 0x2 ref 0 00000100:00100000:2.0:1789760782.983992:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760782.984101:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828864 00000100:00100000:1.0:1789760782.984113:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828864 00000100:00100000:1.0:1789760782.984171:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713084540676 time 713097692203 state 0x0 ref 1 00000100:00100000:1.0:1789760782.984186:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713097952619 time 713097692203 state 0x1 ref 1 00000100:00100000:1.0:1789760782.984261:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713097952619 time 713108867042 state 0x0 ref 1 00000100:00100000:1.0:1789760782.984274:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503828864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760782.984330:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78077952, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253c8 00010000:00010000:1.0:1789760782.984339:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.984354:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760782.984359:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.984365:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.984379:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.989130:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986369 is committed 00002000:00010000:1.0:1789760782.989260:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78077952 -> 78082048 00002000:00010000:1.0:1789760782.989266:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152520 -> 152528 00010000:00010000:1.0:1789760782.989285:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.989297:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.989306:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.989314:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760782.989319:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.989328:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.989346:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.989353:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1143 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78077952->78082047] (req 78077952->78082047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.989575:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503828864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5287us (5576us total) trans 4294986369 rc 0/0 00000100:00100000:1.0:1789760782.989586:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19073 00000100:00100000:1.0:1789760782.989595:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713097952619 time 713108867042 state 0x2 ref 0 00000100:00100000:2.0:1789760782.992661:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760782.992772:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503828992 00000100:00100000:3.0:1789760782.992789:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503828992 00000100:00100000:3.0:1789760782.992866:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713097952619 time 713108867042 state 0x0 ref 1 00000100:00100000:3.0:1789760782.992882:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713109127458 time 713108867042 state 0x1 ref 1 00000100:00100000:3.0:1789760782.992969:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713109127458 time 713117573956 state 0x0 ref 1 00000100:00100000:3.0:1789760782.992988:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760782.993052:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78082048, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253d0 00010000:00010000:3.0:1789760782.993064:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.993083:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760782.993088:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.993096:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.993112:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.003519:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986370 is committed 00002000:00010000:3.0:1789760783.003689:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78082048 -> 78086144 00002000:00010000:3.0:1789760783.003701:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152528 -> 152536 00010000:00010000:3.0:1789760783.003719:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.003733:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.003743:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.003758:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760783.003763:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.003773:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.003798:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.003807:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f114a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78082048->78086143] (req 78082048->78086143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.004005:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503828992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11014us (11344us total) trans 4294986370 rc 0/0 00000100:00100000:3.0:1789760783.004019:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19074 00000100:00100000:3.0:1789760783.004042:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713109127458 time 713117573956 state 0x2 ref 0 00000100:00100000:3.0:1789760783.013193:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.013328:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503829120 00000100:00100000:0.0:1789760783.013343:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503829120 00000100:00100000:0.0:1789760783.013450:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713109127458 time 713117573956 state 0x0 ref 1 00000100:00100000:0.0:1789760783.013470:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713117834372 time 713117573956 state 0x1 ref 1 00000100:00100000:0.0:1789760783.013510:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713117834372 time 713138116130 state 0x0 ref 1 00000100:00100000:0.0:1789760783.013524:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503829120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.013600:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78086144, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253d8 00010000:00010000:0.0:1789760783.013612:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.013631:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.013636:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.013644:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.013659:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.020668:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986371 is committed 00002000:00010000:0.0:1789760783.021196:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78086144 -> 78090240 00002000:00010000:0.0:1789760783.021203:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152536 -> 152544 00010000:00010000:0.0:1789760783.021215:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.021228:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.021237:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.021245:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.021249:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.021258:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.021278:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.021286:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1151 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78086144->78090239] (req 78086144->78090239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.021495:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503829120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7863us (8206us total) trans 4294986371 rc 0/0 00000100:00100000:0.0:1789760783.021508:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19075 00000100:00100000:0.0:1789760783.021518:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713117834372 time 713138116130 state 0x2 ref 0 00000100:00100000:2.0:1789760783.028553:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.028793:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503829248 00000100:00100000:1.0:1789760783.028804:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503829248 00000100:00100000:1.0:1789760783.028869:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713117834372 time 713138116130 state 0x0 ref 1 00000100:00100000:1.0:1789760783.028885:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713138376546 time 713138116130 state 0x1 ref 1 00000100:00100000:1.0:1789760783.028909:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713138376546 time 713153515681 state 0x0 ref 1 00000100:00100000:1.0:1789760783.028920:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503829248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.028968:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78090240, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253e0 00010000:00010000:1.0:1789760783.028976:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.028991:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.028996:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.029002:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.029015:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.036209:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986372 is committed 00002000:00010000:1.0:1789760783.036531:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78090240 -> 78094336 00002000:00010000:1.0:1789760783.036536:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152544 -> 152552 00010000:00010000:1.0:1789760783.036549:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.036583:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.036593:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.036600:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.036604:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.036623:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.036643:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.036651:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1158 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78090240->78094335] (req 78090240->78094335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.036746:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503829248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7821us (8197us total) trans 4294986372 rc 0/0 00000100:00100000:1.0:1789760783.036756:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19076 00000100:00100000:1.0:1789760783.036766:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713138376546 time 713153515681 state 0x2 ref 0 00000100:00100000:2.0:1789760783.040759:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.040882:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503829376 00000100:00100000:1.0:1789760783.040893:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503829376 00000100:00100000:1.0:1789760783.040947:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713138376546 time 713153515681 state 0x0 ref 1 00000100:00100000:1.0:1789760783.040962:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713153776097 time 713153515681 state 0x1 ref 1 00000100:00100000:1.0:1789760783.040993:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713153776097 time 713165600358 state 0x0 ref 1 00000100:00100000:1.0:1789760783.041005:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503829376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.041124:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78094336, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253e8 00010000:00010000:1.0:1789760783.041133:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.041149:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.041154:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.041161:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.041173:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.050060:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986373 is committed 00002000:00010000:1.0:1789760783.050217:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78094336 -> 78098432 00002000:00010000:1.0:1789760783.050223:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152552 -> 152560 00010000:00010000:1.0:1789760783.050236:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.050248:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.050263:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.050271:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.050275:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.050285:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.050306:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.050314:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f115f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78094336->78098431] (req 78094336->78098431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.050463:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503829376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9454us (9709us total) trans 4294986373 rc 0/0 00000100:00100000:1.0:1789760783.050474:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19077 00000100:00100000:1.0:1789760783.050484:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713153776097 time 713165600358 state 0x2 ref 0 00000100:00100000:3.0:1789760783.057539:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.057655:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503829504 00000100:00100000:0.0:1789760783.057666:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503829504 00000100:00100000:0.0:1789760783.057725:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713153776097 time 713165600358 state 0x0 ref 1 00000100:00100000:0.0:1789760783.057748:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713165860774 time 713165600358 state 0x1 ref 1 00000100:00100000:0.0:1789760783.057778:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713165860774 time 713182385250 state 0x0 ref 1 00000100:00100000:0.0:1789760783.057790:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503829504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.057838:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78098432, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253f0 00010000:00010000:0.0:1789760783.057847:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.057861:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.057865:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.057870:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.057883:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.063858:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986374 is committed 00002000:00010000:0.0:1789760783.064036:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78098432 -> 78102528 00002000:00010000:0.0:1789760783.064041:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152560 -> 152568 00010000:00010000:0.0:1789760783.064054:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.064088:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.064096:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.064103:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.064108:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.064118:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.064138:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.064146:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1166 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78098432->78102527] (req 78098432->78102527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.064248:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503829504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6454us (6713us total) trans 4294986374 rc 0/0 00000100:00100000:0.0:1789760783.064259:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19078 00000100:00100000:0.0:1789760783.064269:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713165860774 time 713182385250 state 0x2 ref 0 00000100:00100000:2.0:1789760783.073725:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.073806:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503829632 00000100:00100000:0.0:1789760783.073816:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503829632 00000100:00100000:0.0:1789760783.073866:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713165860774 time 713182385250 state 0x0 ref 1 00000100:00100000:0.0:1789760783.073880:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713182645666 time 713182385250 state 0x1 ref 1 00000100:00100000:0.0:1789760783.073912:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713182645666 time 713198518252 state 0x0 ref 1 00000100:00100000:0.0:1789760783.073923:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503829632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.073987:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78102528, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x253f8 00010000:00010000:0.0:1789760783.073997:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.074012:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.074017:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.074027:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.074041:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.080514:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986375 is committed 00002000:00010000:0.0:1789760783.080634:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78102528 -> 78106624 00002000:00010000:0.0:1789760783.080641:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152568 -> 152576 00010000:00010000:0.0:1789760783.080654:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.080668:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.080684:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.080692:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.080698:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.080707:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.080726:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.080734:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f116d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78102528->78106623] (req 78102528->78106623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.080826:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503829632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6899us (7104us total) trans 4294986375 rc 0/0 00000100:00100000:0.0:1789760783.080836:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19079 00000100:00100000:0.0:1789760783.080846:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713182645666 time 713198518252 state 0x2 ref 0 00000100:00100000:1.0:1789760783.087485:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.087647:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503829760 00000100:00100000:2.0:1789760783.087658:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503829760 00000100:00100000:2.0:1789760783.087713:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713182645666 time 713198518252 state 0x0 ref 1 00000100:00100000:2.0:1789760783.087728:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713198778668 time 713198518252 state 0x1 ref 1 00000100:00100000:2.0:1789760783.087751:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713198778668 time 713212357773 state 0x0 ref 1 00000100:00100000:2.0:1789760783.087762:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503829760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.087808:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78106624, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25400 00010000:00010000:2.0:1789760783.087817:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.087832:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.087836:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.087843:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.087857:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.092295:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986376 is committed 00002000:00010000:2.0:1789760783.092594:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78106624 -> 78110720 00002000:00010000:2.0:1789760783.092663:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152576 -> 152584 00010000:00010000:2.0:1789760783.092678:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.092690:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.092704:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.092711:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.092716:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.092725:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.092745:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.092753:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1174 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78106624->78110719] (req 78106624->78110719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.092929:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503829760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5163us (5448us total) trans 4294986376 rc 0/0 00000100:00100000:2.0:1789760783.092940:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19080 00000100:00100000:2.0:1789760783.092949:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713198778668 time 713212357773 state 0x2 ref 0 00000100:00100000:3.0:1789760783.095973:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.096051:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503829888 00000100:00100000:1.0:1789760783.096060:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503829888 00000100:00100000:1.0:1789760783.096107:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713198778668 time 713212357773 state 0x0 ref 1 00000100:00100000:1.0:1789760783.096125:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713212618189 time 713212357773 state 0x1 ref 1 00000100:00100000:1.0:1789760783.096159:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713212618189 time 713220764958 state 0x0 ref 1 00000100:00100000:1.0:1789760783.096172:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503829888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.096217:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78110720, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25408 00010000:00010000:1.0:1789760783.096226:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.096240:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.096244:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.096251:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.096265:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.100705:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986377 is committed 00002000:00010000:1.0:1789760783.101759:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78110720 -> 78114816 00002000:00010000:1.0:1789760783.101766:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152584 -> 152592 00010000:00010000:1.0:1789760783.101779:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.101792:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.101802:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.101809:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.101814:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.101823:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.101842:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.101850:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f117b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78110720->78114815] (req 78110720->78114815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.101993:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503829888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5817us (6022us total) trans 4294986377 rc 0/0 00000100:00100000:1.0:1789760783.102004:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19081 00000100:00100000:1.0:1789760783.102015:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713212618189 time 713220764958 state 0x2 ref 0 00000100:00100000:0.0:1789760783.106828:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760783.106929:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830016 00000100:00100000:3.0:1789760783.106940:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830016 00000100:00100000:3.0:1789760783.107038:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713212618189 time 713220764958 state 0x0 ref 1 00000100:00100000:3.0:1789760783.107055:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713221025374 time 713220764958 state 0x1 ref 1 00000100:00100000:3.0:1789760783.107132:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713221025374 time 713231736993 state 0x0 ref 1 00000100:00100000:3.0:1789760783.107144:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760783.107192:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78114816, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25410 00010000:00010000:3.0:1789760783.107201:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.107217:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760783.107222:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.107229:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.107242:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.112811:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986378 is committed 00002000:00010000:0.0:1789760783.112932:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78114816 -> 78118912 00002000:00010000:0.0:1789760783.112937:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152592 -> 152600 00010000:00010000:0.0:1789760783.112950:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.112964:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.112974:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.112981:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.112986:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.112996:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.113017:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.113024:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1182 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78114816->78118911] (req 78114816->78118911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.113134:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5986us (6311us total) trans 4294986378 rc 0/0 00000100:00100000:0.0:1789760783.113145:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19082 00000100:00100000:0.0:1789760783.113155:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713221025374 time 713231736993 state 0x2 ref 0 00000100:00100000:2.0:1789760783.117990:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.118074:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830144 00000100:00100000:0.0:1789760783.118085:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830144 00000100:00100000:0.0:1789760783.118141:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713221025374 time 713231736993 state 0x0 ref 1 00000100:00100000:0.0:1789760783.118156:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713231997409 time 713231736993 state 0x1 ref 1 00000100:00100000:0.0:1789760783.118188:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713231997409 time 713242794271 state 0x0 ref 1 00000100:00100000:0.0:1789760783.118278:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503830144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.118332:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78118912, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25418 00010000:00010000:0.0:1789760783.118346:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.118362:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.118366:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.118373:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.118441:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.127144:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986379 is committed 00002000:00010000:0.0:1789760783.127340:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78118912 -> 78123008 00002000:00010000:0.0:1789760783.127345:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152600 -> 152608 00010000:00010000:0.0:1789760783.127360:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.127372:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.127424:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.127433:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.127439:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.127448:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.127468:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.127476:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1189 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78118912->78123007] (req 78118912->78123007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.127589:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503830144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9384us (9599us total) trans 4294986379 rc 0/0 00000100:00100000:0.0:1789760783.127600:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19083 00000100:00100000:0.0:1789760783.127610:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713231997409 time 713242794271 state 0x2 ref 0 00000100:00100000:1.0:1789760783.131977:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.132327:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830272 00000100:00100000:2.0:1789760783.132342:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830272 00000100:00100000:2.0:1789760783.132683:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713231997409 time 713242794271 state 0x0 ref 1 00000100:00100000:2.0:1789760783.132703:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713243054687 time 713242794271 state 0x1 ref 1 00000100:00100000:2.0:1789760783.132797:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713243054687 time 713257403087 state 0x0 ref 1 00000100:00100000:2.0:1789760783.132811:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503830272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.132855:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78123008, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25420 00010000:00010000:2.0:1789760783.132864:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.132879:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.132883:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.132889:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.132902:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.138440:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986380 is committed 00002000:00010000:1.0:1789760783.138844:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78123008 -> 78127104 00002000:00010000:1.0:1789760783.138850:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152608 -> 152616 00010000:00010000:1.0:1789760783.138864:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.138877:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.138886:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.138893:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.138898:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.138906:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.138924:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.138930:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1190 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78123008->78127103] (req 78123008->78127103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.139213:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503830272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6399us (7240us total) trans 4294986380 rc 0/0 00000100:00100000:1.0:1789760783.139225:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19084 00000100:00100000:1.0:1789760783.139235:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713243054687 time 713257403087 state 0x2 ref 0 00000100:00100000:3.0:1789760783.146148:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.146328:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830400 00000100:00100000:2.0:1789760783.146338:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830400 00000100:00100000:2.0:1789760783.146426:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713243054687 time 713257403087 state 0x0 ref 1 00000100:00100000:2.0:1789760783.146442:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713257663503 time 713257403087 state 0x1 ref 1 00000100:00100000:2.0:1789760783.146464:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713257663503 time 713271071342 state 0x0 ref 1 00000100:00100000:2.0:1789760783.146792:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.146835:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78127104, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25428 00010000:00010000:2.0:1789760783.146843:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.146856:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.146860:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.146866:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.146881:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.153652:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986381 is committed 00002000:00010000:1.0:1789760783.153828:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78127104 -> 78131200 00002000:00010000:1.0:1789760783.153832:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152616 -> 152624 00010000:00010000:1.0:1789760783.153845:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.153863:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.153872:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.153879:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.153883:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.153892:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.153912:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.153919:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1197 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78127104->78131199] (req 78127104->78131199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.154008:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7526us (7865us total) trans 4294986381 rc 0/0 00000100:00100000:1.0:1789760783.154019:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19085 00000100:00100000:1.0:1789760783.154030:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713257663503 time 713271071342 state 0x2 ref 0 00000100:00100000:2.0:1789760783.159135:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.159241:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830528 00000100:00100000:0.0:1789760783.159251:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830528 00000100:00100000:0.0:1789760783.159318:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713257663503 time 713271071342 state 0x0 ref 1 00000100:00100000:0.0:1789760783.159333:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713271331758 time 713271071342 state 0x1 ref 1 00000100:00100000:0.0:1789760783.159358:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713271331758 time 713283965608 state 0x0 ref 1 00000100:00100000:0.0:1789760783.159370:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.159457:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78131200, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25430 00010000:00010000:0.0:1789760783.159467:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.159493:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.159498:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.159504:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.159517:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.169244:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986382 is committed 00002000:00010000:0.0:1789760783.169492:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78131200 -> 78135296 00002000:00010000:0.0:1789760783.169497:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152624 -> 152632 00010000:00010000:0.0:1789760783.169510:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.169522:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.169531:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.169543:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.169548:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.169567:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.169589:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.169598:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f119e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78131200->78135295] (req 78131200->78135295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.169799:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10424us (10667us total) trans 4294986382 rc 0/0 00000100:00100000:1.0:1789760783.169811:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19086 00000100:00100000:1.0:1789760783.169821:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713271331758 time 713283965608 state 0x2 ref 0 00000100:00100000:3.0:1789760783.172686:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.173437:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830656 00000100:00100000:0.0:1789760783.173448:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830656 00000100:00100000:0.0:1789760783.173519:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713271331758 time 713283965608 state 0x0 ref 1 00000100:00100000:0.0:1789760783.173536:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713284226024 time 713283965608 state 0x1 ref 1 00000100:00100000:0.0:1789760783.173569:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713284226024 time 713298165304 state 0x0 ref 1 00000100:00100000:0.0:1789760783.173582:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.173646:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78135296, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25438 00010000:00010000:0.0:1789760783.173655:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.173676:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.173681:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.173687:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.173700:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.180915:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986383 is committed 00002000:00010000:0.0:1789760783.181022:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78135296 -> 78139392 00002000:00010000:0.0:1789760783.181028:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152632 -> 152640 00010000:00010000:0.0:1789760783.181041:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.181054:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.181063:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.181071:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.181076:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.181085:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.181111:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.181119:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78135296->78139391] (req 78135296->78139391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.181213:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7628us (8530us total) trans 4294986383 rc 0/0 00000100:00100000:0.0:1789760783.181224:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19087 00000100:00100000:0.0:1789760783.181233:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713284226024 time 713298165304 state 0x2 ref 0 00000100:00100000:3.0:1789760783.185237:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.185770:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830784 00000100:00100000:1.0:1789760783.185783:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830784 00000100:00100000:1.0:1789760783.185845:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713284226024 time 713298165304 state 0x0 ref 1 00000100:00100000:1.0:1789760783.185862:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713298425720 time 713298165304 state 0x1 ref 1 00000100:00100000:1.0:1789760783.185902:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713298425720 time 713310508090 state 0x0 ref 1 00000100:00100000:1.0:1789760783.185915:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.185962:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78139392, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25440 00010000:00010000:1.0:1789760783.185971:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.185987:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.185992:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.185999:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.186013:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.196772:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986384 is committed 00002000:00010000:1.0:1789760783.196887:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78139392 -> 78143488 00002000:00010000:1.0:1789760783.196892:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152640 -> 152648 00010000:00010000:1.0:1789760783.196905:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.196917:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.196925:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.196932:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.196937:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.196946:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.196964:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.196970:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78139392->78143487] (req 78139392->78143487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.197061:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11144us (11830us total) trans 4294986384 rc 0/0 00000100:00100000:1.0:1789760783.197070:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19088 00000100:00100000:1.0:1789760783.197079:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713298425720 time 713310508090 state 0x2 ref 0 00000100:00100000:2.0:1789760783.201452:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.204349:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503830912 00000100:00100000:1.0:1789760783.204363:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503830912 00000100:00100000:1.0:1789760783.204447:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713298425720 time 713310508090 state 0x0 ref 1 00000100:00100000:1.0:1789760783.204465:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713310768506 time 713310508090 state 0x1 ref 1 00000100:00100000:1.0:1789760783.204546:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713310768506 time 713329151381 state 0x0 ref 1 00000100:00100000:1.0:1789760783.204570:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.204632:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78143488, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25448 00010000:00010000:1.0:1789760783.204641:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.204656:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.204661:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.204667:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.204681:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.218666:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986385 is committed 00002000:00010000:1.0:1789760783.220200:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78143488 -> 78147584 00002000:00010000:1.0:1789760783.220208:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152648 -> 152656 00010000:00010000:1.0:1789760783.220222:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.220242:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.220253:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.220261:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.220265:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.220275:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.220308:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.220317:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78143488->78147583] (req 78143488->78147583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.220467:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503830912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15903us (19020us total) trans 4294986385 rc 0/0 00000100:00100000:1.0:1789760783.220478:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19089 00000100:00100000:1.0:1789760783.220488:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713310768506 time 713329151381 state 0x2 ref 0 00000100:00100000:2.0:1789760783.223204:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760783.223304:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831040 00000100:00100000:3.0:1789760783.223316:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831040 00000100:00100000:3.0:1789760783.223368:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713310768506 time 713329151381 state 0x0 ref 1 00000100:00100000:3.0:1789760783.223383:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713329411797 time 713329151381 state 0x1 ref 1 00000100:00100000:3.0:1789760783.223431:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713329411797 time 713348036949 state 0x0 ref 1 00000100:00100000:3.0:1789760783.223444:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760783.223489:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78147584, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25450 00010000:00010000:3.0:1789760783.223497:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.223512:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760783.223517:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.223524:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.223537:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.242091:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986386 is committed 00002000:00010000:2.0:1789760783.242258:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78147584 -> 78151680 00002000:00010000:2.0:1789760783.242264:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152656 -> 152664 00010000:00010000:2.0:1789760783.242277:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.242298:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.242313:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.242320:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.242325:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.242334:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.242353:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.242361:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f11ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78147584->78151679] (req 78147584->78151679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.242505:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19058us (19305us total) trans 4294986386 rc 0/0 00000100:00100000:2.0:1789760783.242515:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19090 00000100:00100000:2.0:1789760783.242525:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713329411797 time 713348036949 state 0x2 ref 0 00000100:00100000:3.0:1789760783.248478:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.248627:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831168 00000100:00100000:0.0:1789760783.248638:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831168 00000100:00100000:0.0:1789760783.248692:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713329411797 time 713348036949 state 0x0 ref 1 00000100:00100000:0.0:1789760783.248708:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713348297365 time 713348036949 state 0x1 ref 1 00000100:00100000:0.0:1789760783.248740:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713348297365 time 713373346815 state 0x0 ref 1 00000100:00100000:0.0:1789760783.248753:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.248801:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78151680, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25458 00010000:00010000:0.0:1789760783.248811:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.248832:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.248837:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.248843:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.248856:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.257310:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986387 is committed 00002000:00010000:0.0:1789760783.257475:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78151680 -> 78155776 00002000:00010000:0.0:1789760783.257480:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152664 -> 152672 00010000:00010000:0.0:1789760783.257493:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257505:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257514:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257520:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.257524:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257532:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257567:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257575:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78151680->78155775] (req 78151680->78155775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.257670:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8913us (9195us total) trans 4294986387 rc 0/0 00000100:00100000:0.0:1789760783.257681:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19091 00000100:00100000:0.0:1789760783.257690:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713348297365 time 713373346815 state 0x2 ref 0 00000100:00100000:1.0:1789760783.263984:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.264144:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831296 00000100:00100000:2.0:1789760783.264155:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831296 00000100:00100000:2.0:1789760783.264223:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713348297365 time 713373346815 state 0x0 ref 1 00000100:00100000:2.0:1789760783.264242:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713373607231 time 713373346815 state 0x1 ref 1 00000100:00100000:2.0:1789760783.264274:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713373607231 time 713388880366 state 0x0 ref 1 00000100:00100000:2.0:1789760783.264291:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.264343:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78155776, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25460 00010000:00010000:2.0:1789760783.264353:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.264367:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.264372:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.264378:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.264429:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.278539:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986388 is committed 00002000:00010000:1.0:1789760783.279209:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78155776 -> 78159872 00002000:00010000:1.0:1789760783.279216:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152672 -> 152680 00010000:00010000:1.0:1789760783.279229:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.279243:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.279252:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.279260:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.279264:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.279273:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.279498:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.279506:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f11c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78155776->78159871] (req 78155776->78159871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.279616:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15327us (15636us total) trans 4294986388 rc 0/0 00000100:00100000:1.0:1789760783.279626:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19092 00000100:00100000:1.0:1789760783.279637:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713373607231 time 713388880366 state 0x2 ref 0 00000100:00100000:3.0:1789760783.286782:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.286887:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831424 00000100:00100000:1.0:1789760783.286897:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831424 00000100:00100000:1.0:1789760783.286951:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713373607231 time 713388880366 state 0x0 ref 1 00000100:00100000:1.0:1789760783.286966:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713389140782 time 713388880366 state 0x1 ref 1 00000100:00100000:1.0:1789760783.287171:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713389140782 time 713411775738 state 0x0 ref 1 00000100:00100000:1.0:1789760783.287185:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.287238:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78159872, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25468 00010000:00010000:1.0:1789760783.287247:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.287262:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.287266:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.287273:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.287296:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.296082:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986389 is committed 00002000:00010000:1.0:1789760783.296272:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78159872 -> 78163968 00002000:00010000:1.0:1789760783.296335:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152680 -> 152688 00010000:00010000:1.0:1789760783.296349:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.296366:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.296375:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.296382:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.296386:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.296414:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.296434:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.296441:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78159872->78163967] (req 78159872->78163967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.296747:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9558us (9968us total) trans 4294986389 rc 0/0 00000100:00100000:1.0:1789760783.296758:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19093 00000100:00100000:1.0:1789760783.296767:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713389140782 time 713411775738 state 0x2 ref 0 00000100:00100000:3.0:1789760783.304835:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.305868:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831552 00000100:00100000:1.0:1789760783.305881:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831552 00000100:00100000:1.0:1789760783.305938:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713389140782 time 713411775738 state 0x0 ref 1 00000100:00100000:1.0:1789760783.305953:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713412036154 time 713411775738 state 0x1 ref 1 00000100:00100000:1.0:1789760783.306038:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713412036154 time 713430643576 state 0x0 ref 1 00000100:00100000:1.0:1789760783.306050:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.306097:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78163968, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25470 00010000:00010000:1.0:1789760783.306106:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.306121:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.306125:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.306130:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.306143:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.317814:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986390 is committed 00002000:00010000:1.0:1789760783.317918:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78163968 -> 78168064 00002000:00010000:1.0:1789760783.317923:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152688 -> 152696 00010000:00010000:1.0:1789760783.317936:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.317949:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.317963:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.317971:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.317976:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.317984:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.318279:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.318288:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f11d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78163968->78168063] (req 78163968->78168063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.318383:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503831552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12330us (13554us total) trans 4294986390 rc 0/0 00000100:00100000:1.0:1789760783.318450:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19094 00000100:00100000:1.0:1789760783.318460:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713412036154 time 713430643576 state 0x2 ref 0 00000100:00100000:2.0:1789760783.326664:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.326755:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831680 00000100:00100000:0.0:1789760783.326765:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831680 00000100:00100000:0.0:1789760783.326833:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713412036154 time 713430643576 state 0x0 ref 1 00000100:00100000:0.0:1789760783.326852:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713430903992 time 713430643576 state 0x1 ref 1 00000100:00100000:0.0:1789760783.326892:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713430903992 time 713451498140 state 0x0 ref 1 00000100:00100000:0.0:1789760783.326904:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.326948:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78168064, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25478 00010000:00010000:0.0:1789760783.326957:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.326972:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.326976:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.326982:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.326995:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.333643:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986391 is committed 00002000:00010000:0.0:1789760783.334327:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78168064 -> 78172160 00002000:00010000:0.0:1789760783.334334:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152696 -> 152704 00010000:00010000:0.0:1789760783.334347:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.334361:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.334369:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.334377:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.334382:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.334437:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.334458:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.334693:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78168064->78172159] (req 78168064->78172159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.334805:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7898us (8145us total) trans 4294986391 rc 0/0 00000100:00100000:0.0:1789760783.334818:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19095 00000100:00100000:0.0:1789760783.334828:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713430903992 time 713451498140 state 0x2 ref 0 00000100:00100000:3.0:1789760783.340342:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.340842:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831808 00000100:00100000:2.0:1789760783.340853:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831808 00000100:00100000:2.0:1789760783.340909:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713430903992 time 713451498140 state 0x0 ref 1 00000100:00100000:2.0:1789760783.340925:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713451758556 time 713451498140 state 0x1 ref 1 00000100:00100000:2.0:1789760783.340959:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713451758556 time 713465565293 state 0x0 ref 1 00000100:00100000:2.0:1789760783.340971:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.341022:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78172160, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25480 00010000:00010000:2.0:1789760783.341032:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.341049:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.341054:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.341061:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.341075:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.347273:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986392 is committed 00002000:00010000:2.0:1789760783.350448:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78172160 -> 78176256 00002000:00010000:2.0:1789760783.350458:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152704 -> 152712 00010000:00010000:2.0:1789760783.350698:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.350712:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.350721:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.350729:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.350734:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.350743:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.350764:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.350771:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78172160->78176255] (req 78172160->78176255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.350940:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9964us (10603us total) trans 4294986392 rc 0/0 00000100:00100000:2.0:1789760783.350950:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19096 00000100:00100000:2.0:1789760783.350960:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713451758556 time 713465565293 state 0x2 ref 0 00000100:00100000:3.0:1789760783.353884:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.354041:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503831936 00000100:00100000:1.0:1789760783.354055:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503831936 00000100:00100000:1.0:1789760783.354119:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713451758556 time 713465565293 state 0x0 ref 1 00000100:00100000:1.0:1789760783.354137:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713465825709 time 713465565293 state 0x1 ref 1 00000100:00100000:1.0:1789760783.354182:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713465825709 time 713478787271 state 0x0 ref 1 00000100:00100000:1.0:1789760783.354194:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.354248:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78176256, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25488 00010000:00010000:1.0:1789760783.354258:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.354276:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.354292:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.354300:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.354315:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.362375:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986393 is committed 00002000:00010000:1.0:1789760783.362920:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78176256 -> 78180352 00002000:00010000:1.0:1789760783.362926:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152712 -> 152720 00010000:00010000:1.0:1789760783.362938:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.362949:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.362957:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.362969:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.362974:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.362982:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.363002:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.363009:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f11eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78176256->78180351] (req 78176256->78180351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.363466:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503831936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9268us (9588us total) trans 4294986393 rc 0/0 00000100:00100000:1.0:1789760783.363476:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19097 00000100:00100000:1.0:1789760783.363486:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713465825709 time 713478787271 state 0x2 ref 0 00000100:00100000:1.0:1789760783.374003:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.374182:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832064 00000100:00100000:2.0:1789760783.374192:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832064 00000100:00100000:2.0:1789760783.374248:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713465825709 time 713478787271 state 0x0 ref 1 00000100:00100000:2.0:1789760783.374263:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713479047687 time 713478787271 state 0x1 ref 1 00000100:00100000:2.0:1789760783.374293:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713479047687 time 713498900001 state 0x0 ref 1 00000100:00100000:2.0:1789760783.374305:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503832064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.374353:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78180352, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25490 00010000:00010000:2.0:1789760783.374362:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.374378:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.374382:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.374601:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.374687:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.384175:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986394 is committed 00002000:00010000:2.0:1789760783.384431:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78180352 -> 78184448 00002000:00010000:2.0:1789760783.384436:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152720 -> 152728 00010000:00010000:2.0:1789760783.384449:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.384461:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.384469:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.384476:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.384480:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.384490:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.384511:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.384518:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f11f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78180352->78184447] (req 78180352->78184447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.384702:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503832064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10393us (10703us total) trans 4294986394 rc 0/0 00000100:00100000:2.0:1789760783.384712:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19098 00000100:00100000:2.0:1789760783.384726:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713479047687 time 713498900001 state 0x2 ref 0 00000100:00100000:3.0:1789760783.389437:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.389501:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832192 00000100:00100000:0.0:1789760783.389510:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832192 00000100:00100000:0.0:1789760783.389578:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713479047687 time 713498900001 state 0x0 ref 1 00000100:00100000:0.0:1789760783.389593:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713499160417 time 713498900001 state 0x1 ref 1 00000100:00100000:0.0:1789760783.389618:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713499160417 time 713514225034 state 0x0 ref 1 00000100:00100000:0.0:1789760783.389639:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.389687:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78184448, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25498 00010000:00010000:0.0:1789760783.389696:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.389712:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.389716:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.389722:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.389735:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.397014:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986395 is committed 00002000:00010000:2.0:1789760783.401889:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78184448 -> 78188544 00002000:00010000:2.0:1789760783.401898:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152728 -> 152736 00010000:00010000:2.0:1789760783.401912:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.401924:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.401933:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.401940:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.401944:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.401959:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.401979:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.401987:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f11f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78184448->78188543] (req 78184448->78188543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.402087:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12446us (12657us total) trans 4294986395 rc 0/0 00000100:00100000:2.0:1789760783.402098:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19099 00000100:00100000:2.0:1789760783.402108:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713499160417 time 713514225034 state 0x2 ref 0 00000100:00100000:1.0:1789760783.407160:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.407347:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832320 00000100:00100000:2.0:1789760783.407357:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832320 00000100:00100000:2.0:1789760783.407434:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713499160417 time 713514225034 state 0x0 ref 1 00000100:00100000:2.0:1789760783.407449:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713514485450 time 713514225034 state 0x1 ref 1 00000100:00100000:2.0:1789760783.407516:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713514485450 time 713532122588 state 0x0 ref 1 00000100:00100000:2.0:1789760783.407529:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.407599:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78188544, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254a0 00010000:00010000:2.0:1789760783.407609:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.407633:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.407638:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.407644:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.407656:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.413063:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986396 is committed 00002000:00010000:2.0:1789760783.413247:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78188544 -> 78192640 00002000:00010000:2.0:1789760783.413253:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152736 -> 152744 00010000:00010000:2.0:1789760783.413266:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.413291:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.413301:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.413308:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.413312:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.413322:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.413348:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.413355:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1200 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78188544->78192639] (req 78188544->78192639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.413534:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6002us (6677us total) trans 4294986396 rc 0/0 00000100:00100000:2.0:1789760783.413545:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19100 00000100:00100000:2.0:1789760783.413563:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713514485450 time 713532122588 state 0x2 ref 0 00000100:00100000:1.0:1789760783.419988:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.420084:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832448 00000100:00100000:2.0:1789760783.420094:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832448 00000100:00100000:2.0:1789760783.420206:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713514485450 time 713532122588 state 0x0 ref 1 00000100:00100000:2.0:1789760783.420222:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713532383004 time 713532122588 state 0x1 ref 1 00000100:00100000:2.0:1789760783.420307:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713532383004 time 713544912409 state 0x0 ref 1 00000100:00100000:2.0:1789760783.420321:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.420366:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78192640, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254a8 00010000:00010000:2.0:1789760783.420375:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.420424:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.420430:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.420437:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.420450:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.426230:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986397 is committed 00002000:00010000:2.0:1789760783.426730:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78192640 -> 78196736 00002000:00010000:2.0:1789760783.426737:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152744 -> 152752 00010000:00010000:2.0:1789760783.426751:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.426763:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.426771:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.426778:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.426854:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.426866:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.426886:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.426901:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1207 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78192640->78196735] (req 78192640->78196735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.427150:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6826us (7166us total) trans 4294986397 rc 0/0 00000100:00100000:2.0:1789760783.427161:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19101 00000100:00100000:2.0:1789760783.427171:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713532383004 time 713544912409 state 0x2 ref 0 00000100:00100000:3.0:1789760783.430511:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.430616:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832576 00000100:00100000:1.0:1789760783.430625:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832576 00000100:00100000:1.0:1789760783.430679:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713532383004 time 713544912409 state 0x0 ref 1 00000100:00100000:1.0:1789760783.430694:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713545172825 time 713544912409 state 0x1 ref 1 00000100:00100000:1.0:1789760783.430725:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713545172825 time 713555331527 state 0x0 ref 1 00000100:00100000:1.0:1789760783.430737:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503832576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.430782:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78196736, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254b0 00010000:00010000:1.0:1789760783.430801:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.430817:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.430821:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.430828:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.430840:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.437065:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986398 is committed 00002000:00010000:1.0:1789760783.437186:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78196736 -> 78200832 00002000:00010000:1.0:1789760783.437192:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152752 -> 152760 00010000:00010000:1.0:1789760783.437205:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.437217:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.437225:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.437237:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.437241:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.437250:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.437269:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.437277:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f120e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78196736->78200831] (req 78196736->78200831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.437431:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503832576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6650us (6882us total) trans 4294986398 rc 0/0 00000100:00100000:1.0:1789760783.437442:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19102 00000100:00100000:1.0:1789760783.437452:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713545172825 time 713555331527 state 0x2 ref 0 00000100:00100000:1.0:1789760783.443785:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.445241:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832704 00000100:00100000:0.0:1789760783.445255:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832704 00000100:00100000:0.0:1789760783.445669:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713545172825 time 713555331527 state 0x0 ref 1 00000100:00100000:0.0:1789760783.445686:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713555591943 time 713555331527 state 0x1 ref 1 00000100:00100000:0.0:1789760783.445716:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713555591943 time 713570322369 state 0x0 ref 1 00000100:00100000:0.0:1789760783.445730:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503832704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.445786:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78200832, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254b8 00010000:00010000:0.0:1789760783.445795:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445810:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.445815:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.445821:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.445836:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.450864:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986399 is committed 00002000:00010000:0.0:1789760783.450982:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78200832 -> 78204928 00002000:00010000:0.0:1789760783.450987:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152760 -> 152768 00010000:00010000:0.0:1789760783.451000:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.451013:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.451021:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.451028:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.451033:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.451041:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.451059:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.451065:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1215 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78200832->78204927] (req 78200832->78204927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.451234:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503832704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5501us (7453us total) trans 4294986399 rc 0/0 00000100:00100000:0.0:1789760783.451246:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19103 00000100:00100000:0.0:1789760783.451256:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713555591943 time 713570322369 state 0x2 ref 0 00000100:00100000:2.0:1789760783.455647:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.456416:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832832 00000100:00100000:1.0:1789760783.456426:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832832 00000100:00100000:1.0:1789760783.456492:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713555591943 time 713570322369 state 0x0 ref 1 00000100:00100000:1.0:1789760783.456507:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713570582785 time 713570322369 state 0x1 ref 1 00000100:00100000:1.0:1789760783.456537:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713570582785 time 713581143768 state 0x0 ref 1 00000100:00100000:1.0:1789760783.456550:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc15850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503832832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.456659:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78204928, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254c0 00010000:00010000:1.0:1789760783.456668:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.456684:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.456689:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.456695:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.456710:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.462211:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986400 is committed 00002000:00010000:1.0:1789760783.462573:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78204928 -> 78209024 00002000:00010000:1.0:1789760783.462579:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152768 -> 152776 00010000:00010000:1.0:1789760783.462591:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.462604:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.462613:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.462620:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.462626:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.462636:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.462654:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.462661:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f121c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78204928->78209023] (req 78204928->78209023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.462759:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc15850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503832832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6206us (7126us total) trans 4294986400 rc 0/0 00000100:00100000:1.0:1789760783.462770:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19104 00000100:00100000:1.0:1789760783.462780:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713570582785 time 713581143768 state 0x2 ref 0 00000100:00100000:3.0:1789760783.468627:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.468769:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503832960 00000100:00100000:2.0:1789760783.468779:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503832960 00000100:00100000:2.0:1789760783.468832:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713570582785 time 713581143768 state 0x0 ref 1 00000100:00100000:2.0:1789760783.468847:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713581404184 time 713581143768 state 0x1 ref 1 00000100:00100000:2.0:1789760783.468878:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713581404184 time 713593483887 state 0x0 ref 1 00000100:00100000:2.0:1789760783.468891:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.468934:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78209024, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254c8 00010000:00010000:2.0:1789760783.468943:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.468958:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.468963:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.468970:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.468983:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.473641:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986401 is committed 00002000:00010000:2.0:1789760783.473862:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78209024 -> 78213120 00002000:00010000:2.0:1789760783.473867:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152776 -> 152784 00010000:00010000:2.0:1789760783.473880:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.473893:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.473901:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.473907:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.473911:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.473919:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.473937:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.473948:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1223 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78209024->78213119] (req 78209024->78213119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.474057:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503832960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5165us (5434us total) trans 4294986401 rc 0/0 00000100:00100000:2.0:1789760783.474064:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19105 00000100:00100000:2.0:1789760783.474073:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713581404184 time 713593483887 state 0x2 ref 0 00000100:00100000:3.0:1789760783.479084:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.479667:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833088 00000100:00100000:2.0:1789760783.479678:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833088 00000100:00100000:2.0:1789760783.479734:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713581404184 time 713593483887 state 0x0 ref 1 00000100:00100000:2.0:1789760783.479750:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713593744303 time 713593483887 state 0x1 ref 1 00000100:00100000:2.0:1789760783.479783:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713593744303 time 713604389815 state 0x0 ref 1 00000100:00100000:2.0:1789760783.479796:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.479842:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78213120, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254d0 00010000:00010000:2.0:1789760783.479851:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.479866:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.479871:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.479877:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.479895:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.491386:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986402 is committed 00002000:00010000:1.0:1789760783.491931:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78213120 -> 78217216 00002000:00010000:1.0:1789760783.491936:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152784 -> 152792 00010000:00010000:1.0:1789760783.491950:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.491962:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.491971:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.491984:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.491990:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.491999:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.492018:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.492026:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f122a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78213120->78217215] (req 78213120->78217215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.492175:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12375us (13095us total) trans 4294986402 rc 0/0 00000100:00100000:1.0:1789760783.492185:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19106 00000100:00100000:1.0:1789760783.492195:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713593744303 time 713604389815 state 0x2 ref 0 00000100:00100000:1.0:1789760783.494647:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.494729:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833216 00000100:00100000:0.0:1789760783.494738:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833216 00000100:00100000:0.0:1789760783.494793:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713593744303 time 713604389815 state 0x0 ref 1 00000100:00100000:0.0:1789760783.494807:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713604650231 time 713604389815 state 0x1 ref 1 00000100:00100000:0.0:1789760783.494837:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713604650231 time 713619443360 state 0x0 ref 1 00000100:00100000:0.0:1789760783.494849:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503833216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.494892:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78217216, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254d8 00010000:00010000:0.0:1789760783.494901:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.494916:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.494920:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.494926:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.494940:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.500037:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986403 is committed 00002000:00010000:2.0:1789760783.500184:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78217216 -> 78221312 00002000:00010000:2.0:1789760783.500189:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152792 -> 152800 00010000:00010000:2.0:1789760783.500202:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.500214:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.500222:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.500230:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.500235:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.500244:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.500263:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.500284:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1231 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78217216->78221311] (req 78217216->78221311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.500506:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503833216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5654us (5862us total) trans 4294986403 rc 0/0 00000100:00100000:2.0:1789760783.500517:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19107 00000100:00100000:2.0:1789760783.500527:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713604650231 time 713619443360 state 0x2 ref 0 00000100:00100000:3.0:1789760783.503883:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.504179:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833344 00000100:00100000:1.0:1789760783.504190:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833344 00000100:00100000:1.0:1789760783.504245:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713604650231 time 713619443360 state 0x0 ref 1 00000100:00100000:1.0:1789760783.504260:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713619703776 time 713619443360 state 0x1 ref 1 00000100:00100000:1.0:1789760783.504330:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713619703776 time 713628936494 state 0x0 ref 1 00000100:00100000:1.0:1789760783.504343:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.504419:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78221312, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254e0 00010000:00010000:1.0:1789760783.504430:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.504447:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.504451:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.504458:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.504472:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.511439:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986404 is committed 00002000:00010000:2.0:1789760783.511568:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78221312 -> 78225408 00002000:00010000:2.0:1789760783.511573:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152800 -> 152808 00010000:00010000:2.0:1789760783.511586:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.511597:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.511605:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.511611:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.511622:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.511630:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.511652:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.511659:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1238 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78221312->78225407] (req 78221312->78225407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.511745:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7399us (7866us total) trans 4294986404 rc 0/0 00000100:00100000:2.0:1789760783.511756:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19108 00000100:00100000:2.0:1789760783.511766:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713619703776 time 713628936494 state 0x2 ref 0 00000100:00100000:3.0:1789760783.516379:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.516762:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833472 00000100:00100000:2.0:1789760783.516772:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833472 00000100:00100000:2.0:1789760783.516825:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713619703776 time 713628936494 state 0x0 ref 1 00000100:00100000:2.0:1789760783.516839:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713629196910 time 713628936494 state 0x1 ref 1 00000100:00100000:2.0:1789760783.516906:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713629196910 time 713641511937 state 0x0 ref 1 00000100:00100000:2.0:1789760783.516919:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.517026:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78225408, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254e8 00010000:00010000:2.0:1789760783.517036:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.517052:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.517057:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.517064:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.517078:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.525808:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986405 is committed 00002000:00010000:1.0:1789760783.525958:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78225408 -> 78229504 00002000:00010000:1.0:1789760783.525963:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152808 -> 152816 00010000:00010000:1.0:1789760783.525976:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.525987:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.526064:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.526073:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.526078:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.526087:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.526107:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.526114:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f123f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78225408->78229503] (req 78225408->78229503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.526332:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9411us (9957us total) trans 4294986405 rc 0/0 00000100:00100000:1.0:1789760783.526343:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19109 00000100:00100000:1.0:1789760783.526353:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713629196910 time 713641511937 state 0x2 ref 0 00000100:00100000:1.0:1789760783.532479:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.532569:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833600 00000100:00100000:2.0:1789760783.532601:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833600 00000100:00100000:2.0:1789760783.532667:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713629196910 time 713641511937 state 0x0 ref 1 00000100:00100000:2.0:1789760783.532683:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713641772353 time 713641511937 state 0x1 ref 1 00000100:00100000:2.0:1789760783.532710:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713641772353 time 713657316508 state 0x0 ref 1 00000100:00100000:2.0:1789760783.532722:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.532773:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78229504, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254f0 00010000:00010000:2.0:1789760783.532782:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.532799:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.532803:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.532809:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.532823:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.537765:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986406 is committed 00002000:00010000:2.0:1789760783.537876:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78229504 -> 78233600 00002000:00010000:2.0:1789760783.537879:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152816 -> 152824 00010000:00010000:2.0:1789760783.537890:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.537899:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.537907:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.537914:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.537918:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.537926:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.537944:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.537952:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1246 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78229504->78233599] (req 78229504->78233599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.538032:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5309us (5559us total) trans 4294986406 rc 0/0 00000100:00100000:2.0:1789760783.538039:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19110 00000100:00100000:2.0:1789760783.538047:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713641772353 time 713657316508 state 0x2 ref 0 00000100:00100000:3.0:1789760783.545794:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.545902:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833728 00000100:00100000:0.0:1789760783.545914:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833728 00000100:00100000:0.0:1789760783.545969:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713641772353 time 713657316508 state 0x0 ref 1 00000100:00100000:0.0:1789760783.545983:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713657576924 time 713657316508 state 0x1 ref 1 00000100:00100000:0.0:1789760783.546056:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713657576924 time 713670662610 state 0x0 ref 1 00000100:00100000:0.0:1789760783.546068:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503833728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.546116:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78233600, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x254f8 00010000:00010000:0.0:1789760783.546126:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.546141:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.546146:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.546152:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.546167:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.550246:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986407 is committed 00002000:00010000:0.0:1789760783.550622:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78233600 -> 78237696 00002000:00010000:0.0:1789760783.550627:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152824 -> 152832 00010000:00010000:0.0:1789760783.550641:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550653:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550663:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550670:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.550674:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550683:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550703:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550710:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f124d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78233600->78237695] (req 78233600->78237695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.550845:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503833728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4774us (5056us total) trans 4294986407 rc 0/0 00000100:00100000:0.0:1789760783.550855:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19111 00000100:00100000:0.0:1789760783.550865:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713657576924 time 713670662610 state 0x2 ref 0 00000100:00100000:3.0:1789760783.554129:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.554575:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833856 00000100:00100000:1.0:1789760783.554586:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833856 00000100:00100000:1.0:1789760783.554651:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713657576924 time 713670662610 state 0x0 ref 1 00000100:00100000:1.0:1789760783.554677:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713670923026 time 713670662610 state 0x1 ref 1 00000100:00100000:1.0:1789760783.554709:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713670923026 time 713679316019 state 0x0 ref 1 00000100:00100000:1.0:1789760783.554721:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.554768:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78237696, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25500 00010000:00010000:1.0:1789760783.554788:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.554804:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.554808:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.554815:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.554828:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.559476:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986408 is committed 00002000:00010000:1.0:1789760783.559916:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78237696 -> 78241792 00002000:00010000:1.0:1789760783.559921:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152832 -> 152840 00010000:00010000:1.0:1789760783.559941:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.559952:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.559960:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.559966:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.559971:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.559984:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.560002:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.560009:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1254 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78237696->78241791] (req 78237696->78241791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.560247:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503833856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5522us (6122us total) trans 4294986408 rc 0/0 00000100:00100000:1.0:1789760783.560258:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19112 00000100:00100000:1.0:1789760783.560268:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713670923026 time 713679316019 state 0x2 ref 0 00000100:00100000:3.0:1789760783.564046:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.564789:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503833984 00000100:00100000:2.0:1789760783.564799:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503833984 00000100:00100000:2.0:1789760783.564858:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713670923026 time 713679316019 state 0x0 ref 1 00000100:00100000:2.0:1789760783.564873:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713679576435 time 713679316019 state 0x1 ref 1 00000100:00100000:2.0:1789760783.564896:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713679576435 time 713689503473 state 0x0 ref 1 00000100:00100000:2.0:1789760783.564908:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503833984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.564959:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78241792, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25508 00010000:00010000:2.0:1789760783.564968:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.564983:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.564987:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.564994:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.565007:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.569805:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986409 is committed 00002000:00010000:2.0:1789760783.569891:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78241792 -> 78245888 00002000:00010000:2.0:1789760783.569896:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152840 -> 152848 00010000:00010000:2.0:1789760783.569925:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.569938:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.569947:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.569955:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.569959:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.569968:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.569988:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.569995:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f125b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78241792->78245887] (req 78241792->78245887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.570093:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503833984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5181us (6051us total) trans 4294986409 rc 0/0 00000100:00100000:2.0:1789760783.570104:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19113 00000100:00100000:2.0:1789760783.570114:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713679576435 time 713689503473 state 0x2 ref 0 00000100:00100000:1.0:1789760783.576156:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.576305:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503834112 00000100:00100000:2.0:1789760783.576315:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503834112 00000100:00100000:2.0:1789760783.576365:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713679576435 time 713689503473 state 0x0 ref 1 00000100:00100000:2.0:1789760783.576379:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713689763889 time 713689503473 state 0x1 ref 1 00000100:00100000:2.0:1789760783.576429:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713689763889 time 713701035428 state 0x0 ref 1 00000100:00100000:2.0:1789760783.576441:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503834112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.576482:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78245888, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25510 00010000:00010000:2.0:1789760783.576490:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.576506:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.576510:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.576516:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.576531:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.581352:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986410 is committed 00002000:00010000:2.0:1789760783.581517:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78245888 -> 78249984 00002000:00010000:2.0:1789760783.581522:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152848 -> 152856 00010000:00010000:2.0:1789760783.581535:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.581548:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.581569:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.581576:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.581581:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.581589:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.581608:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.581683:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1262 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78245888->78249983] (req 78245888->78249983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.581784:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503834112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5342us (5630us total) trans 4294986410 rc 0/0 00000100:00100000:2.0:1789760783.581794:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19114 00000100:00100000:2.0:1789760783.581803:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713689763889 time 713701035428 state 0x2 ref 0 00000100:00100000:3.0:1789760783.584797:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.584921:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503834240 00000100:00100000:0.0:1789760783.584932:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503834240 00000100:00100000:0.0:1789760783.584985:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713689763889 time 713701035428 state 0x0 ref 1 00000100:00100000:0.0:1789760783.585006:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713701295844 time 713701035428 state 0x1 ref 1 00000100:00100000:0.0:1789760783.585030:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713701295844 time 713709636955 state 0x0 ref 1 00000100:00100000:0.0:1789760783.585042:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503834240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.585087:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78249984, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25518 00010000:00010000:0.0:1789760783.585095:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.585110:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.585114:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.585119:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.585132:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.589974:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986411 is committed 00002000:00010000:0.0:1789760783.590159:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78249984 -> 78254080 00002000:00010000:0.0:1789760783.590165:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152856 -> 152864 00010000:00010000:0.0:1789760783.590178:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.590189:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.590197:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.590208:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.590212:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.590221:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.590239:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.590246:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1269 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78249984->78254079] (req 78249984->78254079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.590345:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503834240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5300us (5554us total) trans 4294986411 rc 0/0 00000100:00100000:0.0:1789760783.590354:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19115 00000100:00100000:0.0:1789760783.590363:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713701295844 time 713709636955 state 0x2 ref 0 00000100:00100000:3.0:1789760783.594316:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.594708:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503834368 00000100:00100000:1.0:1789760783.594717:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503834368 00000100:00100000:1.0:1789760783.594770:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713701295844 time 713709636955 state 0x0 ref 1 00000100:00100000:1.0:1789760783.594785:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713709897371 time 713709636955 state 0x1 ref 1 00000100:00100000:1.0:1789760783.594818:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713709897371 time 713719424611 state 0x0 ref 1 00000100:00100000:1.0:1789760783.594834:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503834368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.594880:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78254080, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25520 00010000:00010000:1.0:1789760783.594889:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.594905:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.594910:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.594915:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.594929:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.600785:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986412 is committed 00002000:00010000:2.0:1789760783.600981:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78254080 -> 78258176 00002000:00010000:2.0:1789760783.600987:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152864 -> 152872 00010000:00010000:2.0:1789760783.600999:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.601013:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.601022:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.601030:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.601035:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.601044:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.601065:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.601073:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1270 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78254080->78258175] (req 78254080->78258175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.601164:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503834368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6327us (6854us total) trans 4294986412 rc 0/0 00000100:00100000:2.0:1789760783.601178:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19116 00000100:00100000:2.0:1789760783.601187:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713709897371 time 713719424611 state 0x2 ref 0 00000100:00100000:1.0:1789760783.604022:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.604096:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503834496 00000100:00100000:2.0:1789760783.604106:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503834496 00000100:00100000:2.0:1789760783.604159:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713709897371 time 713719424611 state 0x0 ref 1 00000100:00100000:2.0:1789760783.604174:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713719685027 time 713719424611 state 0x1 ref 1 00000100:00100000:2.0:1789760783.604208:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713719685027 time 713728814355 state 0x0 ref 1 00000100:00100000:2.0:1789760783.604221:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503834496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.604269:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78258176, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25528 00010000:00010000:2.0:1789760783.604279:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.604466:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.604471:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.604477:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.604492:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.609162:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986413 is committed 00002000:00010000:2.0:1789760783.609347:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78258176 -> 78262272 00002000:00010000:2.0:1789760783.609352:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152872 -> 152880 00010000:00010000:2.0:1789760783.609367:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.609379:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.609387:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.609413:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.609418:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.609428:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.609451:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.609459:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1277 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78258176->78262271] (req 78258176->78262271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.609621:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503834496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5384us (5591us total) trans 4294986413 rc 0/0 00000100:00100000:2.0:1789760783.609633:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19117 00000100:00100000:2.0:1789760783.609643:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713719685027 time 713728814355 state 0x2 ref 0 00000100:00100000:3.0:1789760783.612955:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.613193:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503834624 00000100:00100000:2.0:1789760783.613204:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503834624 00000100:00100000:2.0:1789760783.613260:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713719685027 time 713728814355 state 0x0 ref 1 00000100:00100000:2.0:1789760783.613286:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713729074771 time 713728814355 state 0x1 ref 1 00000100:00100000:2.0:1789760783.613319:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713729074771 time 713737925673 state 0x0 ref 1 00000100:00100000:2.0:1789760783.613331:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503834624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.613378:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78262272, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25530 00010000:00010000:2.0:1789760783.613416:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.613433:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.613438:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.613444:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.613458:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.617333:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986414 is committed 00002000:00010000:2.0:1789760783.618455:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78262272 -> 78266368 00002000:00010000:2.0:1789760783.618463:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152880 -> 152888 00010000:00010000:2.0:1789760783.618475:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.618493:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.618502:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.618509:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.618514:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.618528:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.618548:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.618567:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f127e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78262272->78266367] (req 78262272->78266367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.618745:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503834624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5411us (5794us total) trans 4294986414 rc 0/0 00000100:00100000:2.0:1789760783.618755:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19118 00000100:00100000:2.0:1789760783.618766:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713729074771 time 713737925673 state 0x2 ref 0 00000100:00100000:3.0:1789760783.621273:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.621498:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503834752 00000100:00100000:0.0:1789760783.621507:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503834752 00000100:00100000:0.0:1789760783.621574:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713729074771 time 713737925673 state 0x0 ref 1 00000100:00100000:0.0:1789760783.621588:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713738186089 time 713737925673 state 0x1 ref 1 00000100:00100000:0.0:1789760783.621678:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713738186089 time 713746282768 state 0x0 ref 1 00000100:00100000:0.0:1789760783.621696:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503834752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.621740:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78266368, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25538 00010000:00010000:0.0:1789760783.621750:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.621766:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.621825:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.621833:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.621848:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.627879:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986415 is committed 00002000:00010000:0.0:1789760783.628010:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78266368 -> 78270464 00002000:00010000:0.0:1789760783.628015:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152888 -> 152896 00010000:00010000:0.0:1789760783.628028:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.628041:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.628051:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.628058:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.628063:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.628072:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.628093:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.628102:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1285 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78266368->78270463] (req 78266368->78270463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.628192:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503834752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6495us (6921us total) trans 4294986415 rc 0/0 00000100:00100000:0.0:1789760783.628203:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19119 00000100:00100000:0.0:1789760783.628212:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713738186089 time 713746282768 state 0x2 ref 0 00000100:00100000:1.0:1789760783.631854:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.632335:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503834880 00000100:00100000:2.0:1789760783.632345:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503834880 00000100:00100000:2.0:1789760783.632726:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713738186089 time 713746282768 state 0x0 ref 1 00000100:00100000:2.0:1789760783.632743:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713746543184 time 713746282768 state 0x1 ref 1 00000100:00100000:2.0:1789760783.632775:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713746543184 time 713757381652 state 0x0 ref 1 00000100:00100000:2.0:1789760783.632789:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503834880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.632836:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78270464, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25540 00010000:00010000:2.0:1789760783.632846:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.632861:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.632867:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.632873:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.632887:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.637322:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986416 is committed 00002000:00010000:2.0:1789760783.637530:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78270464 -> 78274560 00002000:00010000:2.0:1789760783.637535:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152896 -> 152904 00010000:00010000:2.0:1789760783.637548:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.637571:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.637586:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.637710:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.637715:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.637725:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.637747:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.637756:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f128c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78270464->78274559] (req 78270464->78274559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.637846:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503834880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5055us (5996us total) trans 4294986416 rc 0/0 00000100:00100000:2.0:1789760783.637856:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19120 00000100:00100000:2.0:1789760783.637867:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713746543184 time 713757381652 state 0x2 ref 0 00000100:00100000:3.0:1789760783.642725:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.642865:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835008 00000100:00100000:2.0:1789760783.642876:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835008 00000100:00100000:2.0:1789760783.642939:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713746543184 time 713757381652 state 0x0 ref 1 00000100:00100000:2.0:1789760783.642956:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713757642068 time 713757381652 state 0x1 ref 1 00000100:00100000:2.0:1789760783.642982:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713757642068 time 713767588972 state 0x0 ref 1 00000100:00100000:2.0:1789760783.643004:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.643050:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78274560, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25548 00010000:00010000:2.0:1789760783.643060:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.643076:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.643080:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.643086:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.643100:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.647112:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986417 is committed 00002000:00010000:2.0:1789760783.647533:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78274560 -> 78278656 00002000:00010000:2.0:1789760783.647538:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152904 -> 152912 00010000:00010000:2.0:1789760783.647550:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.647572:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.647580:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.648053:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.648061:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.648072:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.648096:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.648104:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1293 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78274560->78278655] (req 78274560->78278655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.648324:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5327us (5604us total) trans 4294986417 rc 0/0 00000100:00100000:2.0:1789760783.648335:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19121 00000100:00100000:2.0:1789760783.648346:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713757642068 time 713767588972 state 0x2 ref 0 00000100:00100000:3.0:1789760783.652890:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.654287:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835136 00000100:00100000:1.0:1789760783.654301:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835136 00000100:00100000:1.0:1789760783.654359:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713757642068 time 713767588972 state 0x0 ref 1 00000100:00100000:1.0:1789760783.654375:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713767849388 time 713767588972 state 0x1 ref 1 00000100:00100000:1.0:1789760783.654435:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713767849388 time 713779041444 state 0x0 ref 1 00000100:00100000:1.0:1789760783.654449:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503835136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.654501:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78278656, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25550 00010000:00010000:1.0:1789760783.654509:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654524:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.654528:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.654534:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654548:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.660685:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986418 is committed 00002000:00010000:1.0:1789760783.660809:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78278656 -> 78282752 00002000:00010000:1.0:1789760783.660814:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152912 -> 152920 00010000:00010000:1.0:1789760783.660827:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.660839:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.660847:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.660860:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.660864:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.660873:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.660890:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.660897:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f129a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78278656->78282751] (req 78278656->78282751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.661000:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503835136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6542us (8107us total) trans 4294986418 rc 0/0 00000100:00100000:1.0:1789760783.661011:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19122 00000100:00100000:1.0:1789760783.661022:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713767849388 time 713779041444 state 0x2 ref 0 00000100:00100000:1.0:1789760783.664840:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.665050:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835264 00000100:00100000:0.0:1789760783.665063:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835264 00000100:00100000:0.0:1789760783.665127:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713767849388 time 713779041444 state 0x0 ref 1 00000100:00100000:0.0:1789760783.665144:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713779301860 time 713779041444 state 0x1 ref 1 00000100:00100000:0.0:1789760783.665182:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713779301860 time 713789787091 state 0x0 ref 1 00000100:00100000:0.0:1789760783.665385:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.665569:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78282752, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25558 00010000:00010000:0.0:1789760783.665578:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.666602:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.666611:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.666620:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.666638:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.671330:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986419 is committed 00002000:00010000:0.0:1789760783.671456:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78282752 -> 78286848 00002000:00010000:0.0:1789760783.671461:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152920 -> 152928 00010000:00010000:0.0:1789760783.671475:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671488:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671497:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671504:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.671508:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671517:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671538:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.671546:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78282752->78286847] (req 78282752->78286847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.671689:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6304us (6854us total) trans 4294986419 rc 0/0 00000100:00100000:0.0:1789760783.671700:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19123 00000100:00100000:0.0:1789760783.671710:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713779301860 time 713789787091 state 0x2 ref 0 00000100:00100000:3.0:1789760783.677450:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.677882:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835392 00000100:00100000:2.0:1789760783.677892:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835392 00000100:00100000:2.0:1789760783.677949:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713779301860 time 713789787091 state 0x0 ref 1 00000100:00100000:2.0:1789760783.677965:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713790047507 time 713789787091 state 0x1 ref 1 00000100:00100000:2.0:1789760783.678004:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713790047507 time 713802610499 state 0x0 ref 1 00000100:00100000:2.0:1789760783.678017:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.678064:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78286848, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25560 00010000:00010000:2.0:1789760783.678075:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.678090:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.678095:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.678102:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.678116:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.682446:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986420 is committed 00002000:00010000:2.0:1789760783.685852:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78286848 -> 78290944 00002000:00010000:2.0:1789760783.685861:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152928 -> 152936 00010000:00010000:2.0:1789760783.685874:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.685886:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.685894:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.685907:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.685912:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.685920:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.685940:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.685949:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78286848->78290943] (req 78286848->78290943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.686550:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8528us (9104us total) trans 4294986420 rc 0/0 00000100:00100000:2.0:1789760783.686571:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19124 00000100:00100000:2.0:1789760783.686582:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713790047507 time 713802610499 state 0x2 ref 0 00000100:00100000:3.0:1789760783.689227:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.689285:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835520 00000100:00100000:2.0:1789760783.689296:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835520 00000100:00100000:2.0:1789760783.689350:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713790047507 time 713802610499 state 0x0 ref 1 00000100:00100000:2.0:1789760783.689365:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713802870915 time 713802610499 state 0x1 ref 1 00000100:00100000:2.0:1789760783.689460:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713802870915 time 713814065435 state 0x0 ref 1 00000100:00100000:2.0:1789760783.689476:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503835520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.689524:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78290944, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25568 00010000:00010000:2.0:1789760783.689533:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689550:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.689565:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.689573:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689587:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.693296:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986421 is committed 00002000:00010000:2.0:1789760783.693592:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78290944 -> 78295040 00002000:00010000:2.0:1789760783.693597:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152936 -> 152944 00010000:00010000:2.0:1789760783.693610:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693624:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693634:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693642:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.693648:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693657:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693676:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693683:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78290944->78295039] (req 78290944->78295039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.693780:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503835520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4301us (4560us total) trans 4294986421 rc 0/0 00000100:00100000:2.0:1789760783.693790:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19125 00000100:00100000:2.0:1789760783.693800:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713802870915 time 713814065435 state 0x2 ref 0 00000100:00100000:1.0:1789760783.696181:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.696372:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835648 00000100:00100000:2.0:1789760783.696383:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835648 00000100:00100000:2.0:1789760783.696472:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713802870915 time 713814065435 state 0x0 ref 1 00000100:00100000:2.0:1789760783.696487:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713814325851 time 713814065435 state 0x1 ref 1 00000100:00100000:2.0:1789760783.696516:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713814325851 time 713821123441 state 0x0 ref 1 00000100:00100000:2.0:1789760783.696529:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503835648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.696589:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78295040, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25570 00010000:00010000:2.0:1789760783.696599:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.696615:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.696620:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.696626:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.696638:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.703157:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986422 is committed 00002000:00010000:2.0:1789760783.703460:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78295040 -> 78299136 00002000:00010000:2.0:1789760783.703466:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152944 -> 152952 00010000:00010000:2.0:1789760783.703479:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.703493:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.703504:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.703511:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.703516:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.703527:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.703546:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.703565:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78295040->78299135] (req 78295040->78299135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.703917:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503835648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7384us (7738us total) trans 4294986422 rc 0/0 00000100:00100000:2.0:1789760783.703928:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19126 00000100:00100000:2.0:1789760783.703940:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713814325851 time 713821123441 state 0x2 ref 0 00000100:00100000:3.0:1789760783.707755:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.707817:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835776 00000100:00100000:0.0:1789760783.707825:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835776 00000100:00100000:0.0:1789760783.707871:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713814325851 time 713821123441 state 0x0 ref 1 00000100:00100000:0.0:1789760783.707883:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713821383857 time 713821123441 state 0x1 ref 1 00000100:00100000:0.0:1789760783.707983:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713821383857 time 713832588874 state 0x0 ref 1 00000100:00100000:0.0:1789760783.708007:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.708067:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78299136, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25578 00010000:00010000:0.0:1789760783.708077:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.708097:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.708102:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.708109:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.708122:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.714134:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986423 is committed 00002000:00010000:2.0:1789760783.714289:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78299136 -> 78303232 00002000:00010000:2.0:1789760783.714295:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152952 -> 152960 00010000:00010000:2.0:1789760783.714319:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.714334:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.714344:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.714352:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.714358:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.714373:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.714639:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.714649:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f12bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78299136->78303231] (req 78299136->78303231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.714796:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6787us (7153us total) trans 4294986423 rc 0/0 00000100:00100000:2.0:1789760783.714816:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19127 00000100:00100000:2.0:1789760783.714827:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713821383857 time 713832588874 state 0x2 ref 0 00000100:00100000:3.0:1789760783.720947:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.721174:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503835904 00000100:00100000:1.0:1789760783.721184:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503835904 00000100:00100000:1.0:1789760783.721237:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713821383857 time 713832588874 state 0x0 ref 1 00000100:00100000:1.0:1789760783.721256:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713832849290 time 713832588874 state 0x1 ref 1 00000100:00100000:1.0:1789760783.721283:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713832849290 time 713845889579 state 0x0 ref 1 00000100:00100000:1.0:1789760783.721295:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.721342:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78303232, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25580 00010000:00010000:1.0:1789760783.721353:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.721369:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.721373:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.721379:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.721493:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.726997:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986424 is committed 00002000:00010000:1.0:1789760783.727257:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78303232 -> 78307328 00002000:00010000:1.0:1789760783.727262:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152960 -> 152968 00010000:00010000:1.0:1789760783.727325:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.727339:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.727349:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.727356:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.727361:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.727370:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.727512:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.727522:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f12c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78303232->78307327] (req 78303232->78307327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.727733:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503835904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6434us (6790us total) trans 4294986424 rc 0/0 00000100:00100000:1.0:1789760783.727745:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19128 00000100:00100000:1.0:1789760783.727754:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713832849290 time 713845889579 state 0x2 ref 0 00000100:00100000:1.0:1789760783.731947:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.732178:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836032 00000100:00100000:2.0:1789760783.732189:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836032 00000100:00100000:2.0:1789760783.732244:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713832849290 time 713845889579 state 0x0 ref 1 00000100:00100000:2.0:1789760783.732260:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713846149995 time 713845889579 state 0x1 ref 1 00000100:00100000:2.0:1789760783.732292:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713846149995 time 713856898899 state 0x0 ref 1 00000100:00100000:2.0:1789760783.732307:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.732357:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78307328, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25588 00010000:00010000:2.0:1789760783.732366:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732382:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.732386:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.732422:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732438:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.737025:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986425 is committed 00002000:00010000:2.0:1789760783.739190:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78307328 -> 78311424 00002000:00010000:2.0:1789760783.739199:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152968 -> 152976 00010000:00010000:2.0:1789760783.739212:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.739230:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.739238:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.739246:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.739251:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.739258:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.739282:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.739289:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f12cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78307328->78311423] (req 78307328->78311423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.739582:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7273us (7639us total) trans 4294986425 rc 0/0 00000100:00100000:2.0:1789760783.739593:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19129 00000100:00100000:2.0:1789760783.739602:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713846149995 time 713856898899 state 0x2 ref 0 00000100:00100000:3.0:1789760783.742979:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.743038:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836160 00000100:00100000:2.0:1789760783.743048:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836160 00000100:00100000:2.0:1789760783.743117:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713846149995 time 713856898899 state 0x0 ref 1 00000100:00100000:2.0:1789760783.743132:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713857159315 time 713856898899 state 0x1 ref 1 00000100:00100000:2.0:1789760783.743164:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713857159315 time 713867770676 state 0x0 ref 1 00000100:00100000:2.0:1789760783.743175:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503836160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.743223:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78311424, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25590 00010000:00010000:2.0:1789760783.743231:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.743247:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.743253:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.743260:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.743274:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.747981:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986426 is committed 00002000:00010000:2.0:1789760783.748108:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78311424 -> 78315520 00002000:00010000:2.0:1789760783.748113:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152976 -> 152984 00010000:00010000:2.0:1789760783.748126:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.748139:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.748148:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.748156:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.748160:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.748169:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.748190:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.748203:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78311424->78315519] (req 78311424->78315519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.748297:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503836160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5118us (5321us total) trans 4294986426 rc 0/0 00000100:00100000:2.0:1789760783.748308:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19130 00000100:00100000:2.0:1789760783.748319:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713857159315 time 713867770676 state 0x2 ref 0 00000100:00100000:3.0:1789760783.752938:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.753024:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836288 00000100:00100000:0.0:1789760783.753043:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836288 00000100:00100000:0.0:1789760783.753502:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713857159315 time 713867770676 state 0x0 ref 1 00000100:00100000:0.0:1789760783.753515:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713868031092 time 713867770676 state 0x1 ref 1 00000100:00100000:0.0:1789760783.753541:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713868031092 time 713878147769 state 0x0 ref 1 00000100:00100000:0.0:1789760783.753572:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.753623:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78315520, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x25598 00010000:00010000:0.0:1789760783.753636:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753652:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.753657:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.753664:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753678:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.759727:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986427 is committed 00002000:00010000:0.0:1789760783.759876:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78315520 -> 78319616 00002000:00010000:0.0:1789760783.759882:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152984 -> 152992 00010000:00010000:0.0:1789760783.759966:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.759981:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.759991:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.759999:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.760004:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.760013:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.760034:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.760048:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78315520->78319615] (req 78315520->78319615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.760192:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6635us (7258us total) trans 4294986427 rc 0/0 00000100:00100000:0.0:1789760783.760203:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19131 00000100:00100000:0.0:1789760783.760214:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713868031092 time 713878147769 state 0x2 ref 0 00000100:00100000:1.0:1789760783.767812:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.767948:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836416 00000100:00100000:2.0:1789760783.767959:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836416 00000100:00100000:2.0:1789760783.768017:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713868031092 time 713878147769 state 0x0 ref 1 00000100:00100000:2.0:1789760783.768032:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713878408185 time 713878147769 state 0x1 ref 1 00000100:00100000:2.0:1789760783.768065:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713878408185 time 713892670685 state 0x0 ref 1 00000100:00100000:2.0:1789760783.768078:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503836416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.768126:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78319616, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x255a0 00010000:00010000:2.0:1789760783.768136:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.768152:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.768157:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.768163:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.768181:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.772184:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986428 is committed 00002000:00010000:2.0:1789760783.772519:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78319616 -> 78323712 00002000:00010000:2.0:1789760783.772525:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 152992 -> 153000 00010000:00010000:2.0:1789760783.772538:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.772859:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.772870:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.772877:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.772883:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.772893:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.772912:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.772920:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78319616->78323711] (req 78319616->78323711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.773028:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503836416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4945us (5218us total) trans 4294986428 rc 0/0 00000100:00100000:2.0:1789760783.773041:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19132 00000100:00100000:2.0:1789760783.773050:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713878408185 time 713892670685 state 0x2 ref 0 00000100:00100000:3.0:1789760783.777224:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.777302:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836544 00000100:00100000:2.0:1789760783.777312:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836544 00000100:00100000:2.0:1789760783.777365:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713878408185 time 713892670685 state 0x0 ref 1 00000100:00100000:2.0:1789760783.777381:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713892931101 time 713892670685 state 0x1 ref 1 00000100:00100000:2.0:1789760783.777600:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713892931101 time 713902197995 state 0x0 ref 1 00000100:00100000:2.0:1789760783.777804:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503836544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.777849:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78323712, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x255a8 00010000:00010000:2.0:1789760783.777857:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.777872:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.777877:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.777883:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.777897:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.783660:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986429 is committed 00002000:00010000:2.0:1789760783.783780:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78323712 -> 78327808 00002000:00010000:2.0:1789760783.783784:0:8434:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:2.0:1789760783.783787:0:8434:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00002000:00010000:2.0:1789760783.783789:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153000 -> 153008 00010000:00010000:2.0:1789760783.783803:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.783815:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.783823:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.783830:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.783835:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.783844:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.783863:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.783870:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78323712->78327807] (req 78323712->78327807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.784634:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503836544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6826us (7407us total) trans 4294986429 rc 0/0 00000100:00100000:2.0:1789760783.784647:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19133 00000100:00100000:2.0:1789760783.784657:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713892931101 time 713902197995 state 0x2 ref 0 00000100:00100000:3.0:1789760783.788817:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.788932:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836672 00000100:00100000:1.0:1789760783.788944:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836672 00000100:00100000:1.0:1789760783.788998:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713892931101 time 713902197995 state 0x0 ref 1 00000100:00100000:1.0:1789760783.789013:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713902458411 time 713902197995 state 0x1 ref 1 00000100:00100000:1.0:1789760783.789048:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713902458411 time 713913653514 state 0x0 ref 1 00000100:00100000:1.0:1789760783.789060:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.789112:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78327808, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255b0 00010000:00010000:1.0:1789760783.789121:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789136:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.789140:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.789147:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.789161:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.793198:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986430 is committed 00002000:00010000:1.0:1789760783.793318:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78327808 -> 78331904 00002000:00010000:1.0:1789760783.793323:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153008 -> 153016 00010000:00010000:1.0:1789760783.793337:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.793350:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.793364:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.793371:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.793377:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.793386:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.793432:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.793439:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f12ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78327808->78331903] (req 78327808->78331903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.793587:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4525us (4775us total) trans 4294986430 rc 0/0 00000100:00100000:1.0:1789760783.793607:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19134 00000100:00100000:1.0:1789760783.793617:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713902458411 time 713913653514 state 0x2 ref 0 00000100:00100000:1.0:1789760783.796758:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.796855:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836800 00000100:00100000:0.0:1789760783.796866:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836800 00000100:00100000:0.0:1789760783.796993:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713902458411 time 713913653514 state 0x0 ref 1 00000100:00100000:0.0:1789760783.797010:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713913913930 time 713913653514 state 0x1 ref 1 00000100:00100000:0.0:1789760783.797079:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713913913930 time 713921684885 state 0x0 ref 1 00000100:00100000:0.0:1789760783.797093:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503836800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.797141:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78331904, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255b8 00010000:00010000:0.0:1789760783.797150:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.797167:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.797171:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.797178:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.797197:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.800715:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986431 is committed 00002000:00010000:0.0:1789760783.800822:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78331904 -> 78336000 00002000:00010000:0.0:1789760783.800827:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153016 -> 153024 00010000:00010000:0.0:1789760783.800840:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.800853:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.800861:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.800867:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.800871:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.800880:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.800897:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.800905:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f12f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78331904->78335999] (req 78331904->78335999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.801002:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503836800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3908us (4250us total) trans 4294986431 rc 0/0 00000100:00100000:0.0:1789760783.801011:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19135 00000100:00100000:0.0:1789760783.801020:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713913913930 time 713921684885 state 0x2 ref 0 00000100:00100000:3.0:1789760783.803678:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.803764:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503836928 00000100:00100000:2.0:1789760783.803773:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503836928 00000100:00100000:2.0:1789760783.803827:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713913913930 time 713921684885 state 0x0 ref 1 00000100:00100000:2.0:1789760783.803844:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713921945301 time 713921684885 state 0x1 ref 1 00000100:00100000:2.0:1789760783.803877:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713921945301 time 713928483584 state 0x0 ref 1 00000100:00100000:2.0:1789760783.803889:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.804058:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78336000, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255c0 00010000:00010000:2.0:1789760783.804067:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.804083:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.804088:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.804094:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.804108:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.812761:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986432 is committed 00002000:00010000:2.0:1789760783.813371:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78336000 -> 78340096 00002000:00010000:2.0:1789760783.813380:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153024 -> 153032 00010000:00010000:2.0:1789760783.813410:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.813424:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.813433:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.813440:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.813445:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.813455:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.813474:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.813482:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f12fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78336000->78340095] (req 78336000->78340095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.813586:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503836928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9694us (9913us total) trans 4294986432 rc 0/0 00000100:00100000:2.0:1789760783.814482:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19136 00000100:00100000:2.0:1789760783.814493:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713921945301 time 713928483584 state 0x2 ref 0 00000100:00100000:3.0:1789760783.820194:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.820295:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837056 00000100:00100000:2.0:1789760783.820308:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837056 00000100:00100000:2.0:1789760783.820366:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713921945301 time 713928483584 state 0x0 ref 1 00000100:00100000:2.0:1789760783.820381:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713928744000 time 713928483584 state 0x1 ref 1 00000100:00100000:2.0:1789760783.820447:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713928744000 time 713945052830 state 0x0 ref 1 00000100:00100000:2.0:1789760783.820465:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503837056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.820513:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78340096, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255c8 00010000:00010000:2.0:1789760783.820522:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.820539:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.820543:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.820550:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.820572:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.828025:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986433 is committed 00002000:00010000:2.0:1789760783.828138:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78340096 -> 78344192 00002000:00010000:2.0:1789760783.828144:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153032 -> 153040 00010000:00010000:2.0:1789760783.828158:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.828170:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.828179:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.828187:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.828192:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.828202:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.828222:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.828229:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1303 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78340096->78344191] (req 78340096->78344191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.828470:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503837056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8003us (8276us total) trans 4294986433 rc 0/0 00000100:00100000:2.0:1789760783.828482:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19137 00000100:00100000:2.0:1789760783.828493:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713928744000 time 713945052830 state 0x2 ref 0 00000100:00100000:1.0:1789760783.832089:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.832234:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837184 00000100:00100000:2.0:1789760783.832243:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837184 00000100:00100000:2.0:1789760783.832308:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713928744000 time 713945052830 state 0x0 ref 1 00000100:00100000:2.0:1789760783.832323:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713945313246 time 713945052830 state 0x1 ref 1 00000100:00100000:2.0:1789760783.832429:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713945313246 time 713957034816 state 0x0 ref 1 00000100:00100000:2.0:1789760783.832442:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.832488:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78344192, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255d0 00010000:00010000:2.0:1789760783.832498:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.832515:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.832520:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.832527:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.832541:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.836478:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986434 is committed 00002000:00010000:3.0:1789760783.836737:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78344192 -> 78348288 00002000:00010000:3.0:1789760783.836742:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153040 -> 153048 00010000:00010000:3.0:1789760783.836755:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.836766:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.836774:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.836781:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760783.836785:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.836793:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.836813:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.836820:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f130a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78344192->78348287] (req 78344192->78348287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.836967:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4522us (4882us total) trans 4294986434 rc 0/0 00000100:00100000:3.0:1789760783.836978:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19138 00000100:00100000:3.0:1789760783.836987:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713945313246 time 713957034816 state 0x2 ref 0 00000100:00100000:1.0:1789760783.840510:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.840671:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837312 00000100:00100000:0.0:1789760783.840681:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837312 00000100:00100000:0.0:1789760783.840735:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713945313246 time 713957034816 state 0x0 ref 1 00000100:00100000:0.0:1789760783.840751:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713957295232 time 713957034816 state 0x1 ref 1 00000100:00100000:0.0:1789760783.840816:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713957295232 time 713965422096 state 0x0 ref 1 00000100:00100000:0.0:1789760783.840829:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503837312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.840875:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78348288, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255d8 00010000:00010000:0.0:1789760783.840884:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.840899:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760783.840902:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.840908:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.840920:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.845156:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986435 is committed 00002000:00010000:0.0:1789760783.845730:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78348288 -> 78352384 00002000:00010000:0.0:1789760783.845736:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153048 -> 153056 00010000:00010000:0.0:1789760783.845748:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.845761:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.845770:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.845777:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.845782:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.845791:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.845813:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.845820:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1311 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78348288->78352383] (req 78348288->78352383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.846063:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503837312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5231us (5559us total) trans 4294986435 rc 0/0 00000100:00100000:0.0:1789760783.846075:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19139 00000100:00100000:0.0:1789760783.846093:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713957295232 time 713965422096 state 0x2 ref 0 00000100:00100000:0.0:1789760783.851753:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760783.851889:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837440 00000100:00100000:1.0:1789760783.851899:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837440 00000100:00100000:1.0:1789760783.851952:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713957295232 time 713965422096 state 0x0 ref 1 00000100:00100000:1.0:1789760783.851968:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713965682512 time 713965422096 state 0x1 ref 1 00000100:00100000:1.0:1789760783.851994:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713965682512 time 713976600639 state 0x0 ref 1 00000100:00100000:1.0:1789760783.852007:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760783.852053:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78352384, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255e0 00010000:00010000:1.0:1789760783.852062:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.852079:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760783.852090:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.852096:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.852109:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.856717:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986436 is committed 00002000:00010000:1.0:1789760783.856826:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78352384 -> 78356480 00002000:00010000:1.0:1789760783.856830:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153056 -> 153064 00010000:00010000:1.0:1789760783.856841:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.856854:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.856859:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.856864:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.856867:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.856872:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.856886:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.856891:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1318 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78352384->78356479] (req 78352384->78356479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.856967:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4958us (5220us total) trans 4294986436 rc 0/0 00000100:00100000:1.0:1789760783.856977:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19140 00000100:00100000:1.0:1789760783.856986:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713965682512 time 713976600639 state 0x2 ref 0 00000100:00100000:1.0:1789760783.859206:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.859350:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837568 00000100:00100000:2.0:1789760783.859360:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837568 00000100:00100000:2.0:1789760783.860049:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713965682512 time 713976600639 state 0x0 ref 1 00000100:00100000:2.0:1789760783.860065:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713976861055 time 713976600639 state 0x1 ref 1 00000100:00100000:2.0:1789760783.860138:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713976861055 time 713984744436 state 0x0 ref 1 00000100:00100000:2.0:1789760783.860155:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503837568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.860201:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78356480, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255e8 00010000:00010000:2.0:1789760783.860210:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.860226:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760783.860230:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.860236:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.860250:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.864071:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986437 is committed 00002000:00010000:2.0:1789760783.868939:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78356480 -> 78360576 00002000:00010000:2.0:1789760783.868949:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153064 -> 153072 00010000:00010000:2.0:1789760783.868962:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.868975:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.868983:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.868995:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.869000:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.869009:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.869028:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.869035:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f131f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78356480->78360575] (req 78356480->78360575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.869191:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503837568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9035us (9986us total) trans 4294986437 rc 0/0 00000100:00100000:2.0:1789760783.869202:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19141 00000100:00100000:2.0:1789760783.869211:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713976861055 time 713984744436 state 0x2 ref 0 00000100:00100000:1.0:1789760783.874026:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760783.874121:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837696 00000100:00100000:3.0:1789760783.874132:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837696 00000100:00100000:3.0:1789760783.874190:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713976861055 time 713984744436 state 0x0 ref 1 00000100:00100000:3.0:1789760783.874204:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713985004852 time 713984744436 state 0x1 ref 1 00000100:00100000:3.0:1789760783.874265:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713985004852 time 713998871143 state 0x0 ref 1 00000100:00100000:3.0:1789760783.874284:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760783.874330:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78360576, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255f0 00010000:00010000:3.0:1789760783.874338:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.874354:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760783.874359:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.874365:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.874380:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.878298:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986438 is committed 00002000:00010000:3.0:1789760783.878447:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78360576 -> 78364672 00002000:00010000:3.0:1789760783.878453:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153072 -> 153080 00010000:00010000:3.0:1789760783.878687:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.878707:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.878717:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.878725:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760783.878730:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.878740:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.878762:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.878770:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1326 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78360576->78364671] (req 78360576->78364671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.878917:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4636us (4897us total) trans 4294986438 rc 0/0 00000100:00100000:3.0:1789760783.878928:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19142 00000100:00100000:3.0:1789760783.878939:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713985004852 time 713998871143 state 0x2 ref 0 00000100:00100000:0.0:1789760783.884315:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.884385:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837824 00000100:00100000:2.0:1789760783.884426:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837824 00000100:00100000:2.0:1789760783.884495:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713985004852 time 713998871143 state 0x0 ref 1 00000100:00100000:2.0:1789760783.884731:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713999131559 time 713998871143 state 0x1 ref 1 00000100:00100000:2.0:1789760783.884760:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713999131559 time 714009366182 state 0x0 ref 1 00000100:00100000:2.0:1789760783.884776:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.884822:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78364672, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x255f8 00010000:00010000:2.0:1789760783.884832:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.884848:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760783.884852:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.884858:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.884871:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.890981:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760783.891635:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986439 is committed 00002000:00010000:0.0:1789760783.891786:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78364672 -> 78368768 00002000:00010000:0.0:1789760783.891792:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153080 -> 153088 00010000:00010000:0.0:1789760783.891806:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.891819:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.891828:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.891836:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.891841:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.891849:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.891868:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.891881:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f132d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78364672->78368767] (req 78364672->78368767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.892017:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503837824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7239us (7707us total) trans 4294986439 rc 0/0 00000100:00100000:0.0:1789760783.892027:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19143 00000100:00100000:0.0:1789760783.892037:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713999131559 time 714009366182 state 0x2 ref 0 00000100:00100000:1.0:1789760783.899730:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.899904:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503837952 00000100:00100000:2.0:1789760783.899915:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503837952 00000100:00100000:2.0:1789760783.899968:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 713999131559 time 714009366182 state 0x0 ref 1 00000100:00100000:2.0:1789760783.899983:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714009626598 time 714009366182 state 0x1 ref 1 00000100:00100000:2.0:1789760783.900052:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714009626598 time 714024658453 state 0x0 ref 1 00000100:00100000:2.0:1789760783.900064:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503837952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.900113:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78368768, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25600 00010000:00010000:2.0:1789760783.900123:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.900140:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760783.900144:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.900150:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.900163:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.905738:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986440 is committed 00002000:00010000:1.0:1789760783.905894:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78368768 -> 78372864 00002000:00010000:1.0:1789760783.905900:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153088 -> 153096 00010000:00010000:1.0:1789760783.905912:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.905924:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.905933:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.905940:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760783.905943:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.905952:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.905970:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.905976:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1334 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78368768->78372863] (req 78368768->78372863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.906105:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503837952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6036us (6379us total) trans 4294986440 rc 0/0 00000100:00100000:1.0:1789760783.906115:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19144 00000100:00100000:1.0:1789760783.906125:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714009626598 time 714024658453 state 0x2 ref 0 00000100:00100000:1.0:1789760783.909975:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.910059:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838080 00000100:00100000:0.0:1789760783.910068:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838080 00000100:00100000:0.0:1789760783.910122:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714009626598 time 714024658453 state 0x0 ref 1 00000100:00100000:0.0:1789760783.910137:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714024918869 time 714024658453 state 0x1 ref 1 00000100:00100000:0.0:1789760783.910169:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714024918869 time 714034776273 state 0x0 ref 1 00000100:00100000:0.0:1789760783.910181:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503838080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.910228:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78372864, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25608 00010000:00010000:0.0:1789760783.910238:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.910253:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760783.910258:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.910265:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.910295:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.915199:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986441 is committed 00002000:00010000:0.0:1789760783.918101:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78372864 -> 78376960 00002000:00010000:0.0:1789760783.918110:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153096 -> 153104 00010000:00010000:0.0:1789760783.918124:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918138:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918146:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918152:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.918157:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918165:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918182:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.918189:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f133b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78372864->78376959] (req 78372864->78376959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.918362:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503838080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8177us (8391us total) trans 4294986441 rc 0/0 00000100:00100000:0.0:1789760783.918372:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19145 00000100:00100000:0.0:1789760783.918382:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714024918869 time 714034776273 state 0x2 ref 0 00000100:00100000:0.0:1789760783.922125:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760783.922229:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838208 00000100:00100000:3.0:1789760783.922239:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838208 00000100:00100000:3.0:1789760783.922345:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714024918869 time 714034776273 state 0x0 ref 1 00000100:00100000:3.0:1789760783.922358:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714035036689 time 714034776273 state 0x1 ref 1 00000100:00100000:3.0:1789760783.922717:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714035036689 time 714046996341 state 0x0 ref 1 00000100:00100000:3.0:1789760783.922734:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760783.922781:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78376960, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25610 00010000:00010000:3.0:1789760783.922790:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.922804:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760783.922809:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.922815:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.922829:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.931921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986442 is committed 00002000:00010000:3.0:1789760783.932446:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78376960 -> 78381056 00002000:00010000:3.0:1789760783.932452:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153104 -> 153112 00010000:00010000:3.0:1789760783.932465:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.932478:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.932486:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.932493:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760783.932498:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.932508:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.932528:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.932536:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1342 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78376960->78381055] (req 78376960->78381055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.932959:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10225us (10839us total) trans 4294986442 rc 0/0 00000100:00100000:3.0:1789760783.932971:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19146 00000100:00100000:3.0:1789760783.932980:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714035036689 time 714046996341 state 0x2 ref 0 00000100:00100000:1.0:1789760783.936936:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.937014:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838336 00000100:00100000:2.0:1789760783.937024:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838336 00000100:00100000:2.0:1789760783.937082:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714035036689 time 714046996341 state 0x0 ref 1 00000100:00100000:2.0:1789760783.937097:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714047256757 time 714046996341 state 0x1 ref 1 00000100:00100000:2.0:1789760783.937130:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714047256757 time 714061735706 state 0x0 ref 1 00000100:00100000:2.0:1789760783.937143:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503838336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.937194:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78381056, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25618 00010000:00010000:2.0:1789760783.937203:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.937218:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760783.937223:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.937229:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.937243:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.942734:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986443 is committed 00002000:00010000:2.0:1789760783.942930:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78381056 -> 78385152 00002000:00010000:2.0:1789760783.942936:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153112 -> 153120 00010000:00010000:2.0:1789760783.942948:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.942960:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.943015:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.943022:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.943026:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.943034:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.943054:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.943061:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1349 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78381056->78385151] (req 78381056->78385151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.943199:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503838336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6053us (6267us total) trans 4294986443 rc 0/0 00000100:00100000:2.0:1789760783.943210:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19147 00000100:00100000:2.0:1789760783.943220:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714047256757 time 714061735706 state 0x2 ref 0 00000100:00100000:1.0:1789760783.949855:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.950429:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838464 00000100:00100000:2.0:1789760783.950441:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838464 00000100:00100000:2.0:1789760783.950498:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714047256757 time 714061735706 state 0x0 ref 1 00000100:00100000:2.0:1789760783.950513:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714061996122 time 714061735706 state 0x1 ref 1 00000100:00100000:2.0:1789760783.950544:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714061996122 time 714075150737 state 0x0 ref 1 00000100:00100000:2.0:1789760783.950577:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503838464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.950842:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78385152, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25620 00010000:00010000:2.0:1789760783.950852:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.950869:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760783.950873:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.950879:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.950892:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.956298:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986444 is committed 00002000:00010000:2.0:1789760783.956445:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78385152 -> 78389248 00002000:00010000:2.0:1789760783.956450:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153120 -> 153128 00010000:00010000:2.0:1789760783.956465:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956476:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956484:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956491:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.956496:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956505:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956523:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956530:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1350 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78385152->78389247] (req 78385152->78389247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.956945:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503838464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6365us (7094us total) trans 4294986444 rc 0/0 00000100:00100000:2.0:1789760783.956958:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19148 00000100:00100000:2.0:1789760783.956967:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714061996122 time 714075150737 state 0x2 ref 0 00000100:00100000:0.0:1789760783.960123:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760783.960198:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838592 00000100:00100000:2.0:1789760783.960207:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838592 00000100:00100000:2.0:1789760783.960259:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714061996122 time 714075150737 state 0x0 ref 1 00000100:00100000:2.0:1789760783.960274:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714075411153 time 714075150737 state 0x1 ref 1 00000100:00100000:2.0:1789760783.960304:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714075411153 time 714084911408 state 0x0 ref 1 00000100:00100000:2.0:1789760783.960312:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503838592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760783.960350:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78389248, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25628 00010000:00010000:2.0:1789760783.960357:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.960369:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760783.960373:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.960377:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.960412:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.966384:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986445 is committed 00002000:00010000:2.0:1789760783.967475:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78389248 -> 78393344 00002000:00010000:2.0:1789760783.967482:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153128 -> 153136 00010000:00010000:2.0:1789760783.967498:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967510:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967518:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967525:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760783.967531:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967539:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967565:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967573:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1357 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78389248->78393343] (req 78389248->78393343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.967942:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503838592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7623us (7823us total) trans 4294986445 rc 0/0 00000100:00100000:2.0:1789760783.967952:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19149 00000100:00100000:2.0:1789760783.967962:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714075411153 time 714084911408 state 0x2 ref 0 00000100:00100000:1.0:1789760783.974572:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760783.975257:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838720 00000100:00100000:3.0:1789760783.975270:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838720 00000100:00100000:3.0:1789760783.975335:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714075411153 time 714084911408 state 0x0 ref 1 00000100:00100000:3.0:1789760783.975350:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714085171824 time 714084911408 state 0x1 ref 1 00000100:00100000:3.0:1789760783.975382:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714085171824 time 714099988735 state 0x0 ref 1 00000100:00100000:3.0:1789760783.975437:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760783.975482:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78393344, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25630 00010000:00010000:3.0:1789760783.975492:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.975509:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760783.975513:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.975520:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.975533:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.982353:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986446 is committed 00002000:00010000:3.0:1789760783.982732:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78393344 -> 78397440 00002000:00010000:3.0:1789760783.982737:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153136 -> 153144 00010000:00010000:3.0:1789760783.982750:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.982764:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.982772:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.982784:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760783.982788:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.982798:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.982816:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.982823:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f135e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78393344->78397439] (req 78393344->78397439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.982973:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7533us (8423us total) trans 4294986446 rc 0/0 00000100:00100000:3.0:1789760783.982983:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19150 00000100:00100000:3.0:1789760783.982994:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714085171824 time 714099988735 state 0x2 ref 0 00000100:00100000:1.0:1789760783.987363:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760783.987995:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838848 00000100:00100000:0.0:1789760783.988006:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838848 00000100:00100000:0.0:1789760783.988076:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714085171824 time 714099988735 state 0x0 ref 1 00000100:00100000:0.0:1789760783.988091:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714100249151 time 714099988735 state 0x1 ref 1 00000100:00100000:0.0:1789760783.988116:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714100249151 time 714112723507 state 0x0 ref 1 00000100:00100000:0.0:1789760783.988128:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760783.988175:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78397440, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25638 00010000:00010000:0.0:1789760783.988184:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.988200:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760783.988204:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.988210:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.988230:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.994229:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986447 is committed 00002000:00010000:0.0:1789760783.994384:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78397440 -> 78401536 00002000:00010000:0.0:1789760783.994412:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153144 -> 153152 00010000:00010000:0.0:1789760783.994427:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.994441:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.994451:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.994459:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760783.994464:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.994474:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.994494:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.994501:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1365 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78397440->78401535] (req 78397440->78401535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.994932:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6798us (7570us total) trans 4294986447 rc 0/0 00000100:00100000:0.0:1789760783.994944:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19151 00000100:00100000:0.0:1789760783.994954:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714100249151 time 714112723507 state 0x2 ref 0 00000100:00100000:0.0:1789760783.999950:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.000179:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503838976 00000100:00100000:2.0:1789760784.000190:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503838976 00000100:00100000:2.0:1789760784.000298:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714100249151 time 714112723507 state 0x0 ref 1 00000100:00100000:2.0:1789760784.000318:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714112983923 time 714112723507 state 0x1 ref 1 00000100:00100000:2.0:1789760784.000351:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714112983923 time 714124958101 state 0x0 ref 1 00000100:00100000:2.0:1789760784.000363:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.000478:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78401536, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25640 00010000:00010000:2.0:1789760784.000488:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.000504:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.000508:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.000513:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.000527:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.008498:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986448 is committed 00002000:00010000:2.0:1789760784.009172:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78401536 -> 78405632 00002000:00010000:2.0:1789760784.009178:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153152 -> 153160 00010000:00010000:2.0:1789760784.009192:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009207:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009216:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009224:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.009228:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009237:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009259:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009267:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f136c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78401536->78405631] (req 78401536->78405631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.009374:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503838976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9007us (9428us total) trans 4294986448 rc 0/0 00000100:00100000:2.0:1789760784.009385:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19152 00000100:00100000:2.0:1789760784.009788:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714112983923 time 714124958101 state 0x2 ref 0 00000100:00100000:1.0:1789760784.018147:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.019287:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503839104 00000100:00100000:2.0:1789760784.019300:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503839104 00000100:00100000:2.0:1789760784.019372:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714112983923 time 714124958101 state 0x0 ref 1 00000100:00100000:2.0:1789760784.019401:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714125218517 time 714124958101 state 0x1 ref 1 00000100:00100000:2.0:1789760784.019433:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714125218517 time 714144040393 state 0x0 ref 1 00000100:00100000:2.0:1789760784.019446:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503839104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.019507:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78405632, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25648 00010000:00010000:2.0:1789760784.019517:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.019533:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.019537:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.019544:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.019577:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760784.019612:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760784.024189:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986449 is committed 00002000:00010000:2.0:1789760784.024324:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78405632 -> 78409728 00002000:00010000:2.0:1789760784.024330:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153160 -> 153168 00010000:00010000:2.0:1789760784.024342:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.024355:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.024370:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.024379:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.024383:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.024428:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.024448:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.024455:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1373 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78405632->78409727] (req 78405632->78409727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.024563:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503839104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5103us (6408us total) trans 4294986449 rc 0/0 00000100:00100000:2.0:1789760784.024574:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19153 00000100:00100000:2.0:1789760784.024584:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714125218517 time 714144040393 state 0x2 ref 0 00000100:00100000:1.0:1789760784.032805:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.033071:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503839232 00000100:00100000:3.0:1789760784.033082:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503839232 00000100:00100000:3.0:1789760784.033148:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714125218517 time 714144040393 state 0x0 ref 1 00000100:00100000:3.0:1789760784.033163:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714144300809 time 714144040393 state 0x1 ref 1 00000100:00100000:3.0:1789760784.033245:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714144300809 time 714157850489 state 0x0 ref 1 00000100:00100000:3.0:1789760784.033260:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.033315:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78409728, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25650 00010000:00010000:3.0:1789760784.033325:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.033340:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.033345:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.033351:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.033364:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.037947:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986450 is committed 00002000:00010000:3.0:1789760784.038118:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78409728 -> 78413824 00002000:00010000:3.0:1789760784.038123:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153168 -> 153176 00010000:00010000:3.0:1789760784.038201:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.038214:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.038223:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.038230:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.038235:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.038244:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.038263:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.038271:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f137a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78409728->78413823] (req 78409728->78413823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.038561:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5290us (5752us total) trans 4294986450 rc 0/0 00000100:00100000:0.0:1789760784.038573:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19154 00000100:00100000:0.0:1789760784.038583:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714144300809 time 714157850489 state 0x2 ref 0 00000100:00100000:0.0:1789760784.041728:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.041895:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503839360 00000100:00100000:1.0:1789760784.041906:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503839360 00000100:00100000:1.0:1789760784.041960:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714144300809 time 714157850489 state 0x0 ref 1 00000100:00100000:1.0:1789760784.041976:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714158110905 time 714157850489 state 0x1 ref 1 00000100:00100000:1.0:1789760784.042081:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714158110905 time 714166686110 state 0x0 ref 1 00000100:00100000:1.0:1789760784.042095:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.042143:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78413824, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25658 00010000:00010000:1.0:1789760784.042153:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.042170:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.042175:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.042182:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.042196:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.049671:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986451 is committed 00002000:00010000:1.0:1789760784.049792:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78413824 -> 78417920 00002000:00010000:1.0:1789760784.049798:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153176 -> 153184 00010000:00010000:1.0:1789760784.049811:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.049824:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.049833:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.049841:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.049846:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.049855:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.049875:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.049883:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1381 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78413824->78417919] (req 78413824->78417919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.049983:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7886us (8259us total) trans 4294986451 rc 0/0 00000100:00100000:1.0:1789760784.049993:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19155 00000100:00100000:1.0:1789760784.050003:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714158110905 time 714166686110 state 0x2 ref 0 00000100:00100000:1.0:1789760784.056044:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.056133:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503839488 00000100:00100000:2.0:1789760784.056145:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503839488 00000100:00100000:2.0:1789760784.056199:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714158110905 time 714166686110 state 0x0 ref 1 00000100:00100000:2.0:1789760784.056215:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714166946526 time 714166686110 state 0x1 ref 1 00000100:00100000:2.0:1789760784.056247:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714166946526 time 714180854131 state 0x0 ref 1 00000100:00100000:2.0:1789760784.056260:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.056320:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78417920, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25660 00010000:00010000:2.0:1789760784.056330:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.056345:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.056349:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.056355:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.056370:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.061708:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986452 is committed 00002000:00010000:2.0:1789760784.071193:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78417920 -> 78422016 00002000:00010000:2.0:1789760784.071557:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153184 -> 153192 00010000:00010000:2.0:1789760784.071574:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071587:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071596:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071608:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.071613:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071622:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071705:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.071715:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1388 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78417920->78422015] (req 78417920->78422015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.072196:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15932us (16155us total) trans 4294986452 rc 0/0 00000100:00100000:2.0:1789760784.072218:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19156 00000100:00100000:2.0:1789760784.072230:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714166946526 time 714180854131 state 0x2 ref 0 00000100:00100000:3.0:1789760784.081452:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.081573:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503839616 00000100:00100000:2.0:1789760784.081584:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503839616 00000100:00100000:2.0:1789760784.081649:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714166946526 time 714180854131 state 0x0 ref 1 00000100:00100000:2.0:1789760784.081664:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714181114547 time 714180854131 state 0x1 ref 1 00000100:00100000:2.0:1789760784.081698:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714181114547 time 714206304054 state 0x0 ref 1 00000100:00100000:2.0:1789760784.081716:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503839616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760784.081910:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78422016, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25668 00010000:00010000:0.0:1789760784.081920:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081935:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760784.081939:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.081945:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.081959:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.082741:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7cf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523610880:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760784.082775:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5e01be1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:afc717a1-11ba-4ef0-ab10-103aa692ca98:3636:1876699523610752:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.082865:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523611008:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760784.082875:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.082905:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.082957:0:8432:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611008 00000100:00100000:1.0:1789760784.082962:0:8432:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611008 00000100:00100000:1.0:1789760784.083011:0:8432:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:1.0:1789760784.083020:0:8432:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523611008:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760784.083053:0:8432:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1789760784.083092:0:8432:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523611008:12345-0@lo:13:osp-pre-1-1.0 Request processed in 73us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.083099:0:8432:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:1.0:1789760784.083144:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523611008:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1789760784.083194:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760784.083231:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.083294:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523610880 00000100:00100000:1.0:1789760784.083299:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523610880 00000100:00100000:1.0:1789760784.083333:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:1.0:1789760784.083340:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523610880:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760784.083351:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00080000:0.0:1789760784.083729:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760784.087544:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523610752 00000100:00100000:0.0:1789760784.087568:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523610752 00000100:00100000:0.0:1789760784.087634:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:0.0:1789760784.087644:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3636:x1876699523610752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.087736:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523610880:12345-0@lo:13:osp-pre-1-0.0 Request processed in 4392us (4501us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.087739:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3636:x1876699523610752:12345-0@lo:400:kworker.0 Request processed in 95us (4863us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760784.087748:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7cf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523610880:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760784.087749:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1789760784.087754:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:0.0:1789760784.087807:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5e01be1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:afc717a1-11ba-4ef0-ab10-103aa692ca98:3636:1876699523610752:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.087916:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5deab69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523611264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.087974:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760784.087977:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5deab6b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3637:1876699523611392:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.088005:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc673aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523611776:0@lo:400:kworker.0 00000100:00100000:2.0:1789760784.088005:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760784.088023:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc673b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523611904:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.088025:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.088040:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc673a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523612288:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.088066:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760784.088066:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3639:1876699523611136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.088099:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5deab6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523611520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.088107:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760784.088122:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.088128:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.088130:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611264 00000100:00100000:0.0:1789760784.088135:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611264 00000100:00100000:3.0:1789760784.088135:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6738000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3639:1876699523611648:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.088150:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc673b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523612032:0@lo:400:kworker.0 00000100:00100000:2.0:1789760784.088165:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611392 00000100:00100000:3.0:1789760784.088169:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.088170:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611520 00000100:00100000:2.0:1789760784.088171:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611392 00000100:00100000:1.0:1789760784.088181:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.088185:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611520 00000100:00100000:0.0:1789760784.088213:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:0.0:1789760784.088220:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3636:x1876699523611264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.088263:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3636:x1876699523611264:12345-0@lo:400:kworker.0 Request processed in 43us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.088270:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:2.0:1789760784.088278:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611776 00000100:00100000:2.0:1789760784.088284:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611776 00000100:00100000:0.0:1789760784.088299:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789760784.088299:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6739f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3639:1876699523612160:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.088308:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523611520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.088325:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.088338:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523611520:12345-0@lo:400:kworker.0 Request processed in 33us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.088345:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789760784.088437:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612288 00000100:00100000:1.0:1789760784.088442:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612288 00000100:00100000:3.0:1789760784.088442:0:8430:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611904 00000100:00100000:2.0:1789760784.088447:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611136 00000100:00100000:2.0:1789760784.088450:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611136 00000100:00100000:3.0:1789760784.088459:0:8430:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611904 00000100:00100000:2.0:1789760784.088461:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523611648 00000100:00100000:1.0:1789760784.088462:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612160 00000100:00100000:1.0:1789760784.088466:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612160 00000100:00100000:2.0:1789760784.088466:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523611648 00000100:00100000:2.0:1789760784.088489:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 975 00000100:00100000:2.0:1789760784.088496:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3637:x1876699523611392:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.088500:0:8430:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612032 00000100:00100000:3.0:1789760784.088505:0:8430:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612032 00000100:00100000:3.0:1789760784.088533:0:8430:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1789760784.088541:0:8430:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523611904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760784.088544:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3637:x1876699523611392:12345-0@lo:400:kworker.0 Request processed in 47us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.088561:0:8490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1789760784.088569:0:8490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523612032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.088587:0:8430:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523611904:12345-0@lo:400:kworker.0 Request processed in 48us (467us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760784.088594:0:8430:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:0.0:1789760784.088665:0:8490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523612032:12345-0@lo:400:kworker.0 Request processed in 97us (482us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.088672:0:8490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000001:00080000:0.0:1789760784.089010:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986453 is committed 00002000:00010000:0.0:1789760784.089061:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78422016 -> 78426112 00002000:00010000:0.0:1789760784.089065:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153192 -> 153200 00010000:00010000:0.0:1789760784.089076:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.089089:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.089098:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.089104:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760784.089109:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.089117:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.089137:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.089145:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f138f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78422016->78426111] (req 78422016->78426111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.089475:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503839616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7760us (8030us total) trans 4294986453 rc 0/0 00000100:00100000:0.0:1789760784.089485:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19157 00000100:00100000:0.0:1789760784.089495:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714181114547 time 714206304054 state 0x2 ref 0 00000100:00100000:1.0:1789760784.089626:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 976 00000100:00100000:1.0:1789760784.089638:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3636:x1876699523611776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.089668:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 977 00000100:00100000:3.0:1789760784.089681:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3639:x1876699523611136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.089687:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3636:x1876699523611776:12345-0@lo:400:kworker.0 Request processed in 51us (1658us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.089695:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 976 00000100:00100000:1.0:1789760784.089715:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 978 00000100:00100000:1.0:1789760784.089723:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3636:x1876699523612288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.089744:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3639:x1876699523611136:12345-0@lo:400:kworker.0 Request processed in 66us (1638us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760784.089753:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 977 00000100:00100000:1.0:1789760784.089760:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3636:x1876699523612288:12345-0@lo:400:kworker.0 Request processed in 37us (1695us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.089767:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 978 00000100:00100000:3.0:1789760784.089774:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 979 00000100:00100000:3.0:1789760784.089784:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3639:x1876699523611648:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.089789:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 980 00000100:00100000:1.0:1789760784.089799:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3639:x1876699523612160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.089820:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3639:x1876699523611648:12345-0@lo:400:kworker.0 Request processed in 38us (1653us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760784.089828:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 979 00000100:00100000:1.0:1789760784.089840:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3639:x1876699523612160:12345-0@lo:400:kworker.0 Request processed in 43us (1514us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.089848:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 980 00000100:00080000:3.0:1789760784.089877:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760784.089883:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760784.089886:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760784.090421:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3639:1876699523611136:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.090432:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6738000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3639:1876699523611648:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.090442:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6739f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3639:1876699523612160:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.090654:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5deab69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523611264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.090671:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc673aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3636:1876699523611776:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.090682:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc673a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3636:1876699523612288:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.090728:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5deab6ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523611520:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.090738:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc673b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523612032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.092137:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5deab6b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3637:1876699523611392:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.092213:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc673b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523611904:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.095027:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.095342:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503839744 00000100:00100000:1.0:1789760784.095352:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503839744 00000100:00100000:1.0:1789760784.095435:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714181114547 time 714206304054 state 0x0 ref 1 00000100:00100000:1.0:1789760784.095453:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714206564470 time 714206304054 state 0x1 ref 1 00000100:00100000:1.0:1789760784.095477:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714206564470 time 714220084454 state 0x0 ref 1 00000100:00100000:1.0:1789760784.095493:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.095539:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78426112, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25670 00010000:00010000:1.0:1789760784.095559:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.095584:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.095589:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.095597:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.095611:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.096937:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 975 00000001:00080000:0.0:1789760784.100777:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986454 is committed 00002000:00010000:1.0:1789760784.101032:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78426112 -> 78430208 00002000:00010000:1.0:1789760784.101038:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153200 -> 153208 00010000:00010000:1.0:1789760784.101050:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.101061:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.101069:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.101076:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.101081:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.101090:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.101108:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.101116:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1396 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78426112->78430207] (req 78426112->78430207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.101209:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503839744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5714us (6188us total) trans 4294986454 rc 0/0 00000100:00100000:1.0:1789760784.101333:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19158 00000100:00100000:1.0:1789760784.101343:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714206564470 time 714220084454 state 0x2 ref 0 00000100:00100000:1.0:1789760784.108880:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760784.110435:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503839872 00000100:00100000:0.0:1789760784.110455:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503839872 00000100:00100000:0.0:1789760784.110514:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714206564470 time 714220084454 state 0x0 ref 1 00000100:00100000:0.0:1789760784.110530:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714220344870 time 714220084454 state 0x1 ref 1 00000100:00100000:0.0:1789760784.110566:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714220344870 time 714235171949 state 0x0 ref 1 00000100:00100000:0.0:1789760784.110580:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503839872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760784.110629:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78430208, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25678 00010000:00010000:0.0:1789760784.110639:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.110655:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760784.110659:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.110665:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.110680:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.116986:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986455 is committed 00002000:00010000:1.0:1789760784.117159:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78430208 -> 78434304 00002000:00010000:1.0:1789760784.117165:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153208 -> 153216 00010000:00010000:1.0:1789760784.117192:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117205:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117214:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117223:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.117228:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117237:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117257:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117266:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f139d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78430208->78434303] (req 78430208->78434303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.117355:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503839872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6774us (8481us total) trans 4294986455 rc 0/0 00000100:00100000:1.0:1789760784.117365:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19159 00000100:00100000:1.0:1789760784.117374:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714220344870 time 714235171949 state 0x2 ref 0 00000100:00100000:3.0:1789760784.120689:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.128117:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840000 00000100:00100000:2.0:1789760784.128132:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840000 00000100:00100000:2.0:1789760784.128194:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714220344870 time 714235171949 state 0x0 ref 1 00000100:00100000:2.0:1789760784.128209:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714235432365 time 714235171949 state 0x1 ref 1 00000100:00100000:2.0:1789760784.128248:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714235432365 time 714252854539 state 0x0 ref 1 00000100:00100000:2.0:1789760784.128261:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503840000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.128306:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78434304, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25680 00010000:00010000:2.0:1789760784.128316:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.128332:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.128337:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.128342:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.128355:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.136023:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986456 is committed 00002000:00010000:2.0:1789760784.136129:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78434304 -> 78438400 00002000:00010000:2.0:1789760784.136134:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153216 -> 153224 00010000:00010000:2.0:1789760784.136147:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.136159:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.136168:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.136181:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.136186:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.136195:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.136214:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.136233:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78434304->78438399] (req 78434304->78438399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.136327:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503840000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8065us (15641us total) trans 4294986456 rc 0/0 00000100:00100000:2.0:1789760784.136338:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19160 00000100:00100000:2.0:1789760784.136348:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714235432365 time 714252854539 state 0x2 ref 0 00000100:00100000:0.0:1789760784.140726:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.140934:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840128 00000100:00100000:3.0:1789760784.140943:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840128 00000100:00100000:3.0:1789760784.141027:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714235432365 time 714252854539 state 0x0 ref 1 00000100:00100000:3.0:1789760784.141041:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714253114955 time 714252854539 state 0x1 ref 1 00000100:00100000:3.0:1789760784.141068:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714253114955 time 714265674878 state 0x0 ref 1 00000100:00100000:3.0:1789760784.141082:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc16850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503840128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.141132:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78438400, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25688 00010000:00010000:3.0:1789760784.141142:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.141157:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.141161:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.141167:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.141180:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.146559:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986457 is committed 00002000:00010000:3.0:1789760784.146874:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78438400 -> 78442496 00002000:00010000:3.0:1789760784.146881:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153224 -> 153232 00010000:00010000:3.0:1789760784.146894:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.146907:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.146916:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.146923:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.146928:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.146938:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.146958:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.146966:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78438400->78442495] (req 78438400->78442495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.147195:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc16850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503840128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6109us (6640us total) trans 4294986457 rc 0/0 00000100:00100000:3.0:1789760784.147206:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19161 00000100:00100000:3.0:1789760784.147215:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714253114955 time 714265674878 state 0x2 ref 0 00000100:00100000:1.0:1789760784.151672:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760784.151858:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840256 00000100:00100000:0.0:1789760784.151867:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840256 00000100:00100000:0.0:1789760784.151918:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714253114955 time 714265674878 state 0x0 ref 1 00000100:00100000:0.0:1789760784.151934:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714265935294 time 714265674878 state 0x1 ref 1 00000100:00100000:0.0:1789760784.151972:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714265935294 time 714276577793 state 0x0 ref 1 00000100:00100000:0.0:1789760784.151985:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503840256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760784.152149:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78442496, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25690 00010000:00010000:0.0:1789760784.152161:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.152182:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760784.152187:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.152193:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.152208:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.156953:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986458 is committed 00002000:00010000:1.0:1789760784.157185:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78442496 -> 78446592 00002000:00010000:1.0:1789760784.157190:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153232 -> 153240 00010000:00010000:1.0:1789760784.157205:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157218:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157227:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157234:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.157238:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157247:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157265:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157331:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f13b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78442496->78446591] (req 78442496->78446591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.157440:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503840256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5452us (5773us total) trans 4294986458 rc 0/0 00000100:00100000:1.0:1789760784.157450:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19162 00000100:00100000:1.0:1789760784.157460:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714265935294 time 714276577793 state 0x2 ref 0 00000100:00100000:0.0:1789760784.161131:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.161212:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840384 00000100:00100000:1.0:1789760784.161353:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840384 00000100:00100000:1.0:1789760784.161485:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714265935294 time 714276577793 state 0x0 ref 1 00000100:00100000:1.0:1789760784.161501:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714276838209 time 714276577793 state 0x1 ref 1 00000100:00100000:1.0:1789760784.161534:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714276838209 time 714286140058 state 0x0 ref 1 00000100:00100000:1.0:1789760784.161547:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503840384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.161617:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78446592, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25698 00010000:00010000:1.0:1789760784.161627:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.161645:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.161649:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.161656:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.161670:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.169192:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986459 is committed 00002000:00010000:0.0:1789760784.170374:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78446592 -> 78450688 00002000:00010000:0.0:1789760784.170382:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153240 -> 153248 00010000:00010000:0.0:1789760784.170526:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.170540:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.170560:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.170568:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760784.170572:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.170583:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.170674:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.170683:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f13b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78446592->78450687] (req 78446592->78450687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.170882:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503840384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9333us (9756us total) trans 4294986459 rc 0/0 00000100:00100000:0.0:1789760784.170893:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19163 00000100:00100000:0.0:1789760784.170904:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714276838209 time 714286140058 state 0x2 ref 0 00000100:00100000:0.0:1789760784.176581:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.176632:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840512 00000100:00100000:2.0:1789760784.176642:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840512 00000100:00100000:2.0:1789760784.176694:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714276838209 time 714286140058 state 0x0 ref 1 00000100:00100000:2.0:1789760784.176708:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714286400474 time 714286140058 state 0x1 ref 1 00000100:00100000:2.0:1789760784.176787:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714286400474 time 714301384437 state 0x0 ref 1 00000100:00100000:2.0:1789760784.176800:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503840512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.176845:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78450688, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256a0 00010000:00010000:2.0:1789760784.176853:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.176869:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.176873:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.176879:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.176893:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.181180:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986460 is committed 00002000:00010000:2.0:1789760784.181303:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78450688 -> 78454784 00002000:00010000:2.0:1789760784.181308:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153248 -> 153256 00010000:00010000:2.0:1789760784.181321:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.181334:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.181344:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.181351:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.181356:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.181366:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.181385:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.181426:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f13c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78450688->78454783] (req 78450688->78454783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.181509:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503840512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4707us (4935us total) trans 4294986460 rc 0/0 00000100:00100000:2.0:1789760784.181519:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19164 00000100:00100000:2.0:1789760784.181530:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714286400474 time 714301384437 state 0x2 ref 0 00000100:00100000:1.0:1789760784.187256:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.187341:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840640 00000100:00100000:3.0:1789760784.187351:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840640 00000100:00100000:3.0:1789760784.187488:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714286400474 time 714301384437 state 0x0 ref 1 00000100:00100000:3.0:1789760784.187503:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714301644853 time 714301384437 state 0x1 ref 1 00000100:00100000:3.0:1789760784.187579:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714301644853 time 714312185241 state 0x0 ref 1 00000100:00100000:3.0:1789760784.187601:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503840640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.187646:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78454784, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256a8 00010000:00010000:3.0:1789760784.187657:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.187673:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.187677:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.187683:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.187696:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.193500:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986461 is committed 00002000:00010000:3.0:1789760784.193620:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78454784 -> 78458880 00002000:00010000:3.0:1789760784.193625:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153256 -> 153264 00010000:00010000:3.0:1789760784.193697:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.193710:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.193719:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.193726:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.193730:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.193738:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.193756:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.193763:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78454784->78458879] (req 78454784->78458879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.193954:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503840640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6359us (6712us total) trans 4294986461 rc 0/0 00000100:00100000:3.0:1789760784.193965:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19165 00000100:00100000:3.0:1789760784.193974:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714301644853 time 714312185241 state 0x2 ref 0 00000100:00100000:0.0:1789760784.197488:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.197670:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840768 00000100:00100000:1.0:1789760784.197680:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840768 00000100:00100000:1.0:1789760784.197734:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714301644853 time 714312185241 state 0x0 ref 1 00000100:00100000:1.0:1789760784.197750:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714312445657 time 714312185241 state 0x1 ref 1 00000100:00100000:1.0:1789760784.197778:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714312445657 time 714322385453 state 0x0 ref 1 00000100:00100000:1.0:1789760784.197791:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503840768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.198012:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78458880, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256b0 00010000:00010000:1.0:1789760784.198021:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.198035:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.198039:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.198046:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.198060:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.214693:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986462 is committed 00002000:00010000:1.0:1789760784.214800:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78458880 -> 78462976 00002000:00010000:1.0:1789760784.214805:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153264 -> 153272 00010000:00010000:1.0:1789760784.214819:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.214832:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.214841:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.214849:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.214853:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.214868:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.214888:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.214895:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78458880->78462975] (req 78458880->78462975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.214990:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503840768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 17197us (17507us total) trans 4294986462 rc 0/0 00000100:00100000:1.0:1789760784.215001:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19166 00000100:00100000:1.0:1789760784.215010:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714312445657 time 714322385453 state 0x2 ref 0 00000100:00100000:0.0:1789760784.220120:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.220451:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503840896 00000100:00100000:2.0:1789760784.220462:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503840896 00000100:00100000:2.0:1789760784.220514:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714312445657 time 714322385453 state 0x0 ref 1 00000100:00100000:2.0:1789760784.220528:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714322645869 time 714322385453 state 0x1 ref 1 00000100:00100000:2.0:1789760784.220607:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714322645869 time 714345213765 state 0x0 ref 1 00000100:00100000:2.0:1789760784.220620:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503840896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.220794:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78462976, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256b8 00010000:00010000:2.0:1789760784.220804:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.220819:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.220824:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.220830:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.220845:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.227918:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986463 is committed 00002000:00010000:2.0:1789760784.228023:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78462976 -> 78467072 00002000:00010000:2.0:1789760784.228028:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153272 -> 153280 00010000:00010000:2.0:1789760784.228042:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.228055:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.228065:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.228073:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.228078:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.228086:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.228106:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.228114:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78462976->78467071] (req 78462976->78467071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.228200:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503840896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7577us (8085us total) trans 4294986463 rc 0/0 00000100:00100000:2.0:1789760784.228210:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19167 00000100:00100000:2.0:1789760784.228220:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714322645869 time 714345213765 state 0x2 ref 0 00000100:00100000:2.0:1789760784.235317:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.235445:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841024 00000100:00100000:3.0:1789760784.235455:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841024 00000100:00100000:3.0:1789760784.235519:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714322645869 time 714345213765 state 0x0 ref 1 00000100:00100000:3.0:1789760784.235536:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714345474181 time 714345213765 state 0x1 ref 1 00000100:00100000:3.0:1789760784.235578:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714345474181 time 714360183793 state 0x0 ref 1 00000100:00100000:3.0:1789760784.235592:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503841024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.235649:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78467072, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256c0 00010000:00010000:3.0:1789760784.235657:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.235673:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.235679:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.235686:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.235700:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.242187:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986464 is committed 00002000:00010000:3.0:1789760784.242317:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78467072 -> 78471168 00002000:00010000:3.0:1789760784.242322:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153280 -> 153288 00010000:00010000:3.0:1789760784.242335:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.242348:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.242358:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.242365:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.242370:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.242381:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.242432:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.242447:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f13dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78467072->78471167] (req 78467072->78471167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.242566:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503841024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6972us (7254us total) trans 4294986464 rc 0/0 00000100:00100000:3.0:1789760784.242576:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19168 00000100:00100000:3.0:1789760784.242586:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714345474181 time 714360183793 state 0x2 ref 0 00000100:00100000:0.0:1789760784.248088:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.248136:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841152 00000100:00100000:3.0:1789760784.248146:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841152 00000100:00100000:3.0:1789760784.248199:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714345474181 time 714360183793 state 0x0 ref 1 00000100:00100000:3.0:1789760784.248215:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714360444209 time 714360183793 state 0x1 ref 1 00000100:00100000:3.0:1789760784.248245:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714360444209 time 714372851181 state 0x0 ref 1 00000100:00100000:3.0:1789760784.248258:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503841152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.248316:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78471168, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256c8 00010000:00010000:3.0:1789760784.248326:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.248341:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.248345:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.248351:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.248371:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.256699:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986465 is committed 00002000:00010000:3.0:1789760784.260020:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78471168 -> 78475264 00002000:00010000:3.0:1789760784.260171:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153288 -> 153296 00010000:00010000:3.0:1789760784.260186:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.260199:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.260209:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.260217:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.260221:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.260229:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.260248:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.260256:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78471168->78475263] (req 78471168->78475263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.261133:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503841152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12872us (13047us total) trans 4294986465 rc 0/0 00000100:00100000:3.0:1789760784.261146:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19169 00000100:00100000:3.0:1789760784.261156:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714360444209 time 714372851181 state 0x2 ref 0 00000100:00100000:0.0:1789760784.276336:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.276490:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841280 00000100:00100000:1.0:1789760784.276501:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841280 00000100:00100000:1.0:1789760784.276564:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714360444209 time 714372851181 state 0x0 ref 1 00000100:00100000:1.0:1789760784.276579:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714373111597 time 714372851181 state 0x1 ref 1 00000100:00100000:1.0:1789760784.276610:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714373111597 time 714401216277 state 0x0 ref 1 00000100:00100000:1.0:1789760784.276649:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503841280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.276714:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78475264, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256d0 00010000:00010000:1.0:1789760784.276725:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.276741:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.276745:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.276751:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.276764:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.285739:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986466 is committed 00002000:00010000:1.0:1789760784.285866:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78475264 -> 78479360 00002000:00010000:1.0:1789760784.285871:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153296 -> 153304 00010000:00010000:1.0:1789760784.285883:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.285896:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.285905:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.285913:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.285917:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.285926:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.285949:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.285957:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f13ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78475264->78479359] (req 78475264->78479359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.286048:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503841280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9406us (9716us total) trans 4294986466 rc 0/0 00000100:00100000:1.0:1789760784.286058:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19170 00000100:00100000:1.0:1789760784.286068:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714373111597 time 714401216277 state 0x2 ref 0 00000100:00100000:2.0:1789760784.294977:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760784.295064:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841408 00000100:00100000:0.0:1789760784.295075:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841408 00000100:00100000:0.0:1789760784.295128:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714373111597 time 714401216277 state 0x0 ref 1 00000100:00100000:0.0:1789760784.295142:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714401476693 time 714401216277 state 0x1 ref 1 00000100:00100000:0.0:1789760784.295173:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714401476693 time 714419779802 state 0x0 ref 1 00000100:00100000:0.0:1789760784.295185:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503841408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760784.295230:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78479360, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256d8 00010000:00010000:0.0:1789760784.295239:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295254:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760784.295258:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.295265:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295288:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.303449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986467 is committed 00002000:00010000:0.0:1789760784.303570:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78479360 -> 78483456 00002000:00010000:0.0:1789760784.303576:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153304 -> 153312 00010000:00010000:0.0:1789760784.303599:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.303612:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.303622:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.303635:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760784.303640:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.303649:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.303667:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.303674:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f13f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78479360->78483455] (req 78479360->78483455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.303821:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503841408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8633us (8850us total) trans 4294986467 rc 0/0 00000100:00100000:0.0:1789760784.303831:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19171 00000100:00100000:0.0:1789760784.303841:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714401476693 time 714419779802 state 0x2 ref 0 00000100:00100000:0.0:1789760784.320430:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.320676:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841536 00000100:00100000:3.0:1789760784.320687:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841536 00000100:00100000:3.0:1789760784.320748:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714401476693 time 714419779802 state 0x0 ref 1 00000100:00100000:3.0:1789760784.320762:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714420040218 time 714419779802 state 0x1 ref 1 00000100:00100000:3.0:1789760784.320792:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714420040218 time 714445399424 state 0x0 ref 1 00000100:00100000:3.0:1789760784.320804:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503841536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.320852:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78483456, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256e0 00010000:00010000:3.0:1789760784.320861:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.320876:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.320881:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.320886:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.320899:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.331116:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986468 is committed 00002000:00010000:3.0:1789760784.331335:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78483456 -> 78487552 00002000:00010000:3.0:1789760784.331341:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153312 -> 153320 00010000:00010000:3.0:1789760784.331354:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.331367:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.331376:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.331384:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.331457:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.331469:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.331488:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.331496:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f13f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78483456->78487551] (req 78483456->78487551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.331633:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503841536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10826us (11245us total) trans 4294986468 rc 0/0 00000100:00100000:3.0:1789760784.331644:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19172 00000100:00100000:3.0:1789760784.331653:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714420040218 time 714445399424 state 0x2 ref 0 00000100:00100000:3.0:1789760784.334612:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.334746:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841664 00000100:00100000:2.0:1789760784.334756:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841664 00000100:00100000:2.0:1789760784.334811:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714420040218 time 714445399424 state 0x0 ref 1 00000100:00100000:2.0:1789760784.334826:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714445659840 time 714445399424 state 0x1 ref 1 00000100:00100000:2.0:1789760784.334858:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714445659840 time 714459464340 state 0x0 ref 1 00000100:00100000:2.0:1789760784.334870:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503841664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.334919:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78487552, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256e8 00010000:00010000:2.0:1789760784.334928:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.334944:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.334949:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.334955:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.334969:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.342026:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986469 is committed 00002000:00010000:2.0:1789760784.342170:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78487552 -> 78491648 00002000:00010000:2.0:1789760784.342176:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153320 -> 153328 00010000:00010000:2.0:1789760784.342189:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342202:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342212:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342219:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.342224:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342238:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342257:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.342264:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f13ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78487552->78491647] (req 78487552->78491647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.342352:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503841664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7480us (7745us total) trans 4294986469 rc 0/0 00000100:00100000:2.0:1789760784.342361:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19173 00000100:00100000:2.0:1789760784.342371:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714445659840 time 714459464340 state 0x2 ref 0 00000100:00100000:2.0:1789760784.346142:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.346460:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841792 00000100:00100000:1.0:1789760784.346471:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841792 00000100:00100000:1.0:1789760784.346539:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714445659840 time 714459464340 state 0x0 ref 1 00000100:00100000:1.0:1789760784.346575:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714459724756 time 714459464340 state 0x1 ref 1 00000100:00100000:1.0:1789760784.346609:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714459724756 time 714471215785 state 0x0 ref 1 00000100:00100000:1.0:1789760784.346622:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503841792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.346664:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78491648, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256f0 00010000:00010000:1.0:1789760784.346673:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.346688:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.346692:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.346703:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.346717:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.354446:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986470 is committed 00002000:00010000:0.0:1789760784.354874:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78491648 -> 78495744 00002000:00010000:0.0:1789760784.354880:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153328 -> 153336 00010000:00010000:0.0:1789760784.354894:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.354911:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.354919:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.354926:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760784.355092:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.355104:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.355125:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.355134:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1406 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78491648->78495743] (req 78491648->78495743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.355267:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503841792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8642us (9130us total) trans 4294986470 rc 0/0 00000100:00100000:0.0:1789760784.355286:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19174 00000100:00100000:0.0:1789760784.355295:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714459724756 time 714471215785 state 0x2 ref 0 00000100:00100000:0.0:1789760784.358916:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.359000:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503841920 00000100:00100000:2.0:1789760784.359011:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503841920 00000100:00100000:2.0:1789760784.359063:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714459724756 time 714471215785 state 0x0 ref 1 00000100:00100000:2.0:1789760784.359078:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714471476201 time 714471215785 state 0x1 ref 1 00000100:00100000:2.0:1789760784.359108:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714471476201 time 714483714437 state 0x0 ref 1 00000100:00100000:2.0:1789760784.359120:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503841920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.359168:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78495744, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x256f8 00010000:00010000:2.0:1789760784.359178:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.359193:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.359198:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.359203:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.359217:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.365583:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986471 is committed 00002000:00010000:2.0:1789760784.365690:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78495744 -> 78499840 00002000:00010000:2.0:1789760784.365695:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153336 -> 153344 00010000:00010000:2.0:1789760784.365708:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.365721:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.365730:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.365738:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.365742:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.365752:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.365772:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.365787:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f140d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78495744->78499839] (req 78495744->78499839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.365918:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503841920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6796us (7008us total) trans 4294986471 rc 0/0 00000100:00100000:2.0:1789760784.365927:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19175 00000100:00100000:2.0:1789760784.365936:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714471476201 time 714483714437 state 0x2 ref 0 00000100:00100000:3.0:1789760784.372088:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.372465:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842048 00000100:00100000:2.0:1789760784.372475:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842048 00000100:00100000:2.0:1789760784.372528:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714471476201 time 714483714437 state 0x0 ref 1 00000100:00100000:2.0:1789760784.372544:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714483974853 time 714483714437 state 0x1 ref 1 00000100:00100000:2.0:1789760784.372585:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714483974853 time 714497191831 state 0x0 ref 1 00000100:00100000:2.0:1789760784.372599:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503842048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.372645:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78499840, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25700 00010000:00010000:2.0:1789760784.372653:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.372668:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.372673:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.372679:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.372692:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.380959:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986472 is committed 00002000:00010000:3.0:1789760784.381156:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78499840 -> 78503936 00002000:00010000:3.0:1789760784.381161:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153344 -> 153352 00010000:00010000:3.0:1789760784.381174:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.381186:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.381194:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.381201:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.381206:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.381215:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.381233:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.381245:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1414 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78499840->78503935] (req 78499840->78503935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.381339:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503842048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8738us (9254us total) trans 4294986472 rc 0/0 00000100:00100000:3.0:1789760784.381349:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19176 00000100:00100000:3.0:1789760784.381359:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714483974853 time 714497191831 state 0x2 ref 0 00000100:00100000:1.0:1789760784.384773:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.384837:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842176 00000100:00100000:3.0:1789760784.384848:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842176 00000100:00100000:3.0:1789760784.384903:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714483974853 time 714497191831 state 0x0 ref 1 00000100:00100000:3.0:1789760784.384918:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714497452247 time 714497191831 state 0x1 ref 1 00000100:00100000:3.0:1789760784.384951:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714497452247 time 714509557335 state 0x0 ref 1 00000100:00100000:3.0:1789760784.384964:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503842176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.385009:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78503936, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25708 00010000:00010000:3.0:1789760784.385018:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.385033:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.385038:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.385044:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.385062:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.392158:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986473 is committed 00002000:00010000:3.0:1789760784.392293:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78503936 -> 78508032 00002000:00010000:3.0:1789760784.392298:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153352 -> 153360 00010000:00010000:3.0:1789760784.392310:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.392322:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.392331:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.392338:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.392343:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.392353:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.392372:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.392380:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f141b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78503936->78508031] (req 78503936->78508031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.392500:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503842176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7533us (7732us total) trans 4294986473 rc 0/0 00000100:00100000:3.0:1789760784.392510:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19177 00000100:00100000:3.0:1789760784.392520:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714497452247 time 714509557335 state 0x2 ref 0 00000100:00100000:0.0:1789760784.397618:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.397700:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842304 00000100:00100000:1.0:1789760784.397709:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842304 00000100:00100000:1.0:1789760784.397764:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714497452247 time 714509557335 state 0x0 ref 1 00000100:00100000:1.0:1789760784.397780:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714509817751 time 714509557335 state 0x1 ref 1 00000100:00100000:1.0:1789760784.397806:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714509817751 time 714522412524 state 0x0 ref 1 00000100:00100000:1.0:1789760784.397821:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.397868:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78508032, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25710 00010000:00010000:1.0:1789760784.397878:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.397894:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.397907:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.397914:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.397932:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.404885:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986474 is committed 00002000:00010000:0.0:1789760784.405011:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78508032 -> 78512128 00002000:00010000:0.0:1789760784.405016:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153360 -> 153368 00010000:00010000:0.0:1789760784.405028:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405040:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405048:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405055:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760784.405060:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405068:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405088:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405095:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1422 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78508032->78512127] (req 78508032->78512127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.405186:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7365us (7573us total) trans 4294986474 rc 0/0 00000100:00100000:0.0:1789760784.405200:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19178 00000100:00100000:0.0:1789760784.405210:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714509817751 time 714522412524 state 0x2 ref 0 00000100:00100000:0.0:1789760784.411173:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.411469:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842432 00000100:00100000:2.0:1789760784.411479:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842432 00000100:00100000:2.0:1789760784.411537:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714509817751 time 714522412524 state 0x0 ref 1 00000100:00100000:2.0:1789760784.411562:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714522672940 time 714522412524 state 0x1 ref 1 00000100:00100000:2.0:1789760784.411594:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714522672940 time 714536200407 state 0x0 ref 1 00000100:00100000:2.0:1789760784.411606:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.411655:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78512128, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25718 00010000:00010000:2.0:1789760784.411664:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.411681:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.411685:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.411692:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.411706:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.416800:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986475 is committed 00002000:00010000:2.0:1789760784.416972:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78512128 -> 78516224 00002000:00010000:2.0:1789760784.416978:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153368 -> 153376 00010000:00010000:2.0:1789760784.416990:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417001:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417010:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417017:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.417022:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417032:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417051:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417058:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1429 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78512128->78516223] (req 78512128->78516223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.417365:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5754us (6306us total) trans 4294986475 rc 0/0 00000100:00100000:2.0:1789760784.417378:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19179 00000100:00100000:2.0:1789760784.417417:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714522672940 time 714536200407 state 0x2 ref 0 00000100:00100000:1.0:1789760784.421230:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.422301:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842560 00000100:00100000:2.0:1789760784.422312:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842560 00000100:00100000:2.0:1789760784.422364:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714522672940 time 714536200407 state 0x0 ref 1 00000100:00100000:2.0:1789760784.422378:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714536460823 time 714536200407 state 0x1 ref 1 00000100:00100000:2.0:1789760784.422421:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714536460823 time 714547027718 state 0x0 ref 1 00000100:00100000:2.0:1789760784.422432:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.422472:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78516224, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25720 00010000:00010000:2.0:1789760784.422481:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.422496:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.422500:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.422506:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.422519:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.427161:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986476 is committed 00002000:00010000:2.0:1789760784.427373:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78516224 -> 78520320 00002000:00010000:2.0:1789760784.427379:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153376 -> 153384 00010000:00010000:2.0:1789760784.427419:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.427432:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.427441:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.427448:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.427453:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.427463:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.427483:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.427491:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1430 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78516224->78520319] (req 78516224->78520319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.427581:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5145us (6357us total) trans 4294986476 rc 0/0 00000100:00100000:2.0:1789760784.427592:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19180 00000100:00100000:2.0:1789760784.427601:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714536460823 time 714547027718 state 0x2 ref 0 00000100:00100000:0.0:1789760784.431442:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.431520:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842688 00000100:00100000:3.0:1789760784.431531:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842688 00000100:00100000:3.0:1789760784.431611:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714536460823 time 714547027718 state 0x0 ref 1 00000100:00100000:3.0:1789760784.431628:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714547288134 time 714547027718 state 0x1 ref 1 00000100:00100000:3.0:1789760784.431659:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714547288134 time 714556265938 state 0x0 ref 1 00000100:00100000:3.0:1789760784.431672:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503842688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.431716:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78520320, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25728 00010000:00010000:3.0:1789760784.431725:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.431741:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.431745:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.431752:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.431765:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.436481:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986477 is committed 00002000:00010000:3.0:1789760784.436618:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78520320 -> 78524416 00002000:00010000:3.0:1789760784.436624:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153384 -> 153392 00010000:00010000:3.0:1789760784.436636:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.436650:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.436659:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.436667:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.436673:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.436681:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.436701:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.436709:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1437 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78520320->78524415] (req 78520320->78524415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.436806:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503842688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5131us (5367us total) trans 4294986477 rc 0/0 00000100:00100000:3.0:1789760784.436817:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19181 00000100:00100000:3.0:1789760784.436827:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714547288134 time 714556265938 state 0x2 ref 0 00000100:00100000:0.0:1789760784.444156:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.444317:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842816 00000100:00100000:1.0:1789760784.444327:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842816 00000100:00100000:1.0:1789760784.444441:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714547288134 time 714556265938 state 0x0 ref 1 00000100:00100000:1.0:1789760784.444455:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714556526354 time 714556265938 state 0x1 ref 1 00000100:00100000:1.0:1789760784.444486:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714556526354 time 714569092173 state 0x0 ref 1 00000100:00100000:1.0:1789760784.444498:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503842816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.444544:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78524416, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25730 00010000:00010000:1.0:1789760784.444583:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.444599:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.444603:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.444609:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.444629:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.449250:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986478 is committed 00002000:00010000:1.0:1789760784.449357:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78524416 -> 78528512 00002000:00010000:1.0:1789760784.449363:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153392 -> 153400 00010000:00010000:1.0:1789760784.449376:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.449421:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.449431:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.449438:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.449443:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.449453:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.449472:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.449479:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f143e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78524416->78528511] (req 78524416->78528511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.449565:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503842816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5053us (5403us total) trans 4294986478 rc 0/0 00000100:00100000:1.0:1789760784.449575:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19182 00000100:00100000:1.0:1789760784.449584:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714556526354 time 714569092173 state 0x2 ref 0 00000100:00100000:1.0:1789760784.453730:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.453819:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503842944 00000100:00100000:2.0:1789760784.453829:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503842944 00000100:00100000:2.0:1789760784.453884:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714556526354 time 714569092173 state 0x0 ref 1 00000100:00100000:2.0:1789760784.453907:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714569352589 time 714569092173 state 0x1 ref 1 00000100:00100000:2.0:1789760784.453938:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714569352589 time 714578544622 state 0x0 ref 1 00000100:00100000:2.0:1789760784.453949:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.453998:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78528512, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25738 00010000:00010000:2.0:1789760784.454006:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.454022:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.454026:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.454031:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.454044:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.458453:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986479 is committed 00002000:00010000:2.0:1789760784.458584:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78528512 -> 78532608 00002000:00010000:2.0:1789760784.458589:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153400 -> 153408 00010000:00010000:2.0:1789760784.458603:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.458621:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.458630:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.458637:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.458641:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.458650:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.458670:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.458678:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1445 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78528512->78532607] (req 78528512->78532607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.458768:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503842944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4816us (5043us total) trans 4294986479 rc 0/0 00000100:00100000:2.0:1789760784.458779:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19183 00000100:00100000:2.0:1789760784.458789:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714569352589 time 714578544622 state 0x2 ref 0 00000100:00100000:0.0:1789760784.464836:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.464973:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843072 00000100:00100000:2.0:1789760784.464983:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843072 00000100:00100000:2.0:1789760784.465039:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714569352589 time 714578544622 state 0x0 ref 1 00000100:00100000:2.0:1789760784.465054:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714578805038 time 714578544622 state 0x1 ref 1 00000100:00100000:2.0:1789760784.465086:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714578805038 time 714589691648 state 0x0 ref 1 00000100:00100000:2.0:1789760784.465098:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.465144:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78532608, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25740 00010000:00010000:2.0:1789760784.465153:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.465168:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.465173:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.465179:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.465191:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.469367:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986480 is committed 00002000:00010000:2.0:1789760784.471968:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78532608 -> 78536704 00002000:00010000:2.0:1789760784.471978:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153408 -> 153416 00010000:00010000:2.0:1789760784.471992:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.472005:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.472019:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.472026:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.472030:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.472038:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.472057:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.472064:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f144c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78532608->78536703] (req 78532608->78536703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.472162:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7062us (7333us total) trans 4294986480 rc 0/0 00000100:00100000:2.0:1789760784.472171:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19184 00000100:00100000:2.0:1789760784.472181:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714578805038 time 714589691648 state 0x2 ref 0 00000100:00100000:0.0:1789760784.475146:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.475259:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843200 00000100:00100000:3.0:1789760784.475271:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843200 00000100:00100000:3.0:1789760784.475326:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714578805038 time 714589691648 state 0x0 ref 1 00000100:00100000:3.0:1789760784.475340:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714589952064 time 714589691648 state 0x1 ref 1 00000100:00100000:3.0:1789760784.475372:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714589952064 time 714599978660 state 0x0 ref 1 00000100:00100000:3.0:1789760784.475384:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503843200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.475506:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78536704, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25748 00010000:00010000:3.0:1789760784.475516:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.475532:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.475537:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.475543:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.476726:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.486676:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986481 is committed 00002000:00010000:3.0:1789760784.486900:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78536704 -> 78540800 00002000:00010000:3.0:1789760784.486906:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153416 -> 153424 00010000:00010000:3.0:1789760784.486919:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.486932:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.486941:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.486948:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.486952:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.486973:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.486993:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.487000:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1453 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78536704->78540799] (req 78536704->78540799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.487158:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503843200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11769us (12013us total) trans 4294986481 rc 0/0 00000100:00100000:3.0:1789760784.487168:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19185 00000100:00100000:3.0:1789760784.487178:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714589952064 time 714599978660 state 0x2 ref 0 00000100:00100000:1.0:1789760784.494582:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.494820:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843328 00000100:00100000:2.0:1789760784.494830:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843328 00000100:00100000:2.0:1789760784.494885:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714589952064 time 714599978660 state 0x0 ref 1 00000100:00100000:2.0:1789760784.494900:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714600239076 time 714599978660 state 0x1 ref 1 00000100:00100000:2.0:1789760784.494933:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714600239076 time 714619540022 state 0x0 ref 1 00000100:00100000:2.0:1789760784.494946:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503843328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.494995:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78540800, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25750 00010000:00010000:2.0:1789760784.495004:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.495019:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.495023:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.495030:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.495044:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.500798:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986482 is committed 00002000:00010000:2.0:1789760784.504681:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78540800 -> 78544896 00002000:00010000:2.0:1789760784.504691:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153424 -> 153432 00010000:00010000:2.0:1789760784.504705:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.504717:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.504725:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.504737:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.504742:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.504751:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.504770:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.504778:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f145a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78540800->78544895] (req 78540800->78544895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.504870:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503843328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9922us (10293us total) trans 4294986482 rc 0/0 00000100:00100000:2.0:1789760784.504881:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19186 00000100:00100000:2.0:1789760784.504890:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714600239076 time 714619540022 state 0x2 ref 0 00000100:00100000:3.0:1789760784.508142:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.508574:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843456 00000100:00100000:2.0:1789760784.508635:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843456 00000100:00100000:2.0:1789760784.508687:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714600239076 time 714619540022 state 0x0 ref 1 00000100:00100000:2.0:1789760784.508701:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714619800438 time 714619540022 state 0x1 ref 1 00000100:00100000:2.0:1789760784.508769:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714619800438 time 714633374434 state 0x0 ref 1 00000100:00100000:2.0:1789760784.508786:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.508842:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78544896, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25758 00010000:00010000:2.0:1789760784.508851:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.508867:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.508871:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.508877:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.508890:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.518155:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986483 is committed 00002000:00010000:2.0:1789760784.518358:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78544896 -> 78548992 00002000:00010000:2.0:1789760784.518364:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153432 -> 153440 00010000:00010000:2.0:1789760784.518377:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518463:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518473:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518480:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.518486:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518495:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518515:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518524:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1461 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78544896->78548991] (req 78544896->78548991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.518716:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9927us (10579us total) trans 4294986483 rc 0/0 00000100:00100000:2.0:1789760784.518727:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19187 00000100:00100000:2.0:1789760784.518738:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714619800438 time 714633374434 state 0x2 ref 0 00000100:00100000:0.0:1789760784.522023:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.522090:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843584 00000100:00100000:2.0:1789760784.522100:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843584 00000100:00100000:2.0:1789760784.522154:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714619800438 time 714633374434 state 0x0 ref 1 00000100:00100000:2.0:1789760784.522168:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714633634850 time 714633374434 state 0x1 ref 1 00000100:00100000:2.0:1789760784.522202:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714633634850 time 714646808120 state 0x0 ref 1 00000100:00100000:2.0:1789760784.522214:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503843584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.522258:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78548992, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25760 00010000:00010000:2.0:1789760784.522269:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.522288:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.522293:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.522299:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.522313:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.528143:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986484 is committed 00002000:00010000:2.0:1789760784.528286:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78548992 -> 78553088 00002000:00010000:2.0:1789760784.528291:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153440 -> 153448 00010000:00010000:2.0:1789760784.528303:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.528315:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.528325:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.528332:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.528337:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.528346:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.528364:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.528372:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1468 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78548992->78553087] (req 78548992->78553087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.528509:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503843584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6292us (6491us total) trans 4294986484 rc 0/0 00000100:00100000:2.0:1789760784.528518:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19188 00000100:00100000:2.0:1789760784.528527:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714633634850 time 714646808120 state 0x2 ref 0 00000100:00100000:1.0:1789760784.532288:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.532370:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843712 00000100:00100000:3.0:1789760784.532379:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843712 00000100:00100000:3.0:1789760784.532459:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714633634850 time 714646808120 state 0x0 ref 1 00000100:00100000:3.0:1789760784.532475:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714647068536 time 714646808120 state 0x1 ref 1 00000100:00100000:3.0:1789760784.532505:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714647068536 time 714657112320 state 0x0 ref 1 00000100:00100000:3.0:1789760784.532518:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.532580:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78553088, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25768 00010000:00010000:3.0:1789760784.532589:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.532605:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.532609:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.532615:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.532629:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.537237:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986485 is committed 00002000:00010000:3.0:1789760784.537363:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78553088 -> 78557184 00002000:00010000:3.0:1789760784.537368:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153448 -> 153456 00010000:00010000:3.0:1789760784.537381:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.537420:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.537429:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.537437:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.537442:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.537451:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.537471:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.537480:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f146f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78553088->78557183] (req 78553088->78557183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.537600:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5079us (5316us total) trans 4294986485 rc 0/0 00000100:00100000:3.0:1789760784.537611:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19189 00000100:00100000:3.0:1789760784.537621:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714647068536 time 714657112320 state 0x2 ref 0 00000100:00100000:3.0:1789760784.540364:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.540484:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843840 00000100:00100000:1.0:1789760784.540494:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843840 00000100:00100000:1.0:1789760784.540562:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714647068536 time 714657112320 state 0x0 ref 1 00000100:00100000:1.0:1789760784.540577:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714657372736 time 714657112320 state 0x1 ref 1 00000100:00100000:1.0:1789760784.540614:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714657372736 time 714665221108 state 0x0 ref 1 00000100:00100000:1.0:1789760784.540627:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.540670:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78557184, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25770 00010000:00010000:1.0:1789760784.540680:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.540700:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.540706:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.540712:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.540725:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.544574:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986486 is committed 00002000:00010000:1.0:1789760784.544741:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78557184 -> 78561280 00002000:00010000:1.0:1789760784.544747:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153456 -> 153464 00010000:00010000:1.0:1789760784.544759:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.544770:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.544779:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.544786:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.544791:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.544799:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.544823:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.544831:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1476 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78557184->78561279] (req 78557184->78561279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.544958:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4328us (4599us total) trans 4294986486 rc 0/0 00000100:00100000:1.0:1789760784.544969:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19190 00000100:00100000:1.0:1789760784.544978:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714657372736 time 714665221108 state 0x2 ref 0 00000100:00100000:0.0:1789760784.548025:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.548109:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503843968 00000100:00100000:2.0:1789760784.548118:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503843968 00000100:00100000:2.0:1789760784.548168:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714657372736 time 714665221108 state 0x0 ref 1 00000100:00100000:2.0:1789760784.548181:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714665481524 time 714665221108 state 0x1 ref 1 00000100:00100000:2.0:1789760784.548210:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714665481524 time 714672816769 state 0x0 ref 1 00000100:00100000:2.0:1789760784.548221:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.548264:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78561280, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25778 00010000:00010000:2.0:1789760784.548282:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.548297:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.548302:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.548308:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.548321:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.552416:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986487 is committed 00002000:00010000:0.0:1789760784.552519:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78561280 -> 78565376 00002000:00010000:0.0:1789760784.552524:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153464 -> 153472 00010000:00010000:0.0:1789760784.552615:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.552628:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.552637:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.552643:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760784.552648:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.552657:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.552678:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.552686:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f147d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78561280->78565375] (req 78561280->78565375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.552783:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503843968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4559us (4761us total) trans 4294986487 rc 0/0 00000100:00100000:0.0:1789760784.552794:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19191 00000100:00100000:0.0:1789760784.552803:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714665481524 time 714672816769 state 0x2 ref 0 00000100:00100000:2.0:1789760784.558584:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760784.558669:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844096 00000100:00100000:0.0:1789760784.558680:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844096 00000100:00100000:0.0:1789760784.558736:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714665481524 time 714672816769 state 0x0 ref 1 00000100:00100000:0.0:1789760784.558751:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714673077185 time 714672816769 state 0x1 ref 1 00000100:00100000:0.0:1789760784.558781:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714673077185 time 714683387753 state 0x0 ref 1 00000100:00100000:0.0:1789760784.558793:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503844096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760784.558849:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78565376, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25780 00010000:00010000:0.0:1789760784.558858:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.558875:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760784.558880:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.558887:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.558901:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.564590:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986488 is committed 00002000:00010000:0.0:1789760784.565280:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78565376 -> 78569472 00002000:00010000:0.0:1789760784.565286:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153472 -> 153480 00010000:00010000:0.0:1789760784.565300:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.565312:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.565321:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.565329:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760784.565334:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.565343:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.565367:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.565375:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1484 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78565376->78569471] (req 78565376->78569471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.565509:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503844096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6712us (6929us total) trans 4294986488 rc 0/0 00000100:00100000:0.0:1789760784.565520:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19192 00000100:00100000:0.0:1789760784.565530:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714673077185 time 714683387753 state 0x2 ref 0 00000100:00100000:3.0:1789760784.568043:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.568204:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844224 00000100:00100000:2.0:1789760784.568215:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844224 00000100:00100000:2.0:1789760784.568267:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714673077185 time 714683387753 state 0x0 ref 1 00000100:00100000:2.0:1789760784.568282:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714683648169 time 714683387753 state 0x1 ref 1 00000100:00100000:2.0:1789760784.568362:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714683648169 time 714692967569 state 0x0 ref 1 00000100:00100000:2.0:1789760784.568377:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503844224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.568456:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78569472, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25788 00010000:00010000:2.0:1789760784.568465:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.568479:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.568483:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.568489:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.568508:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.575717:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986489 is committed 00002000:00010000:2.0:1789760784.575932:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78569472 -> 78573568 00002000:00010000:2.0:1789760784.575937:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153480 -> 153488 00010000:00010000:2.0:1789760784.575949:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.575960:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.575979:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.575986:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.575990:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576000:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576019:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576025:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f148b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78569472->78573567] (req 78569472->78573567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.576723:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503844224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8342us (8683us total) trans 4294986489 rc 0/0 00000100:00100000:2.0:1789760784.576735:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19193 00000100:00100000:2.0:1789760784.576744:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714683648169 time 714692967569 state 0x2 ref 0 00000100:00100000:3.0:1789760784.583160:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.583256:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844352 00000100:00100000:2.0:1789760784.583265:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844352 00000100:00100000:2.0:1789760784.583317:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714683648169 time 714692967569 state 0x0 ref 1 00000100:00100000:2.0:1789760784.583332:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714693227985 time 714692967569 state 0x1 ref 1 00000100:00100000:2.0:1789760784.583364:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714693227985 time 714707970518 state 0x0 ref 1 00000100:00100000:2.0:1789760784.583377:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503844352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.583461:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78573568, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25790 00010000:00010000:2.0:1789760784.583483:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.583498:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.583503:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.583510:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.583524:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.591923:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986490 is committed 00002000:00010000:3.0:1789760784.602526:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78573568 -> 78577664 00002000:00010000:3.0:1789760784.602536:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153488 -> 153496 00010000:00010000:3.0:1789760784.602561:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.602574:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.602582:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.602589:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.602593:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.603101:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.603124:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.603132:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1492 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78573568->78577663] (req 78573568->78577663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.603292:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503844352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19911us (20136us total) trans 4294986490 rc 0/0 00000100:00100000:3.0:1789760784.603302:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19194 00000100:00100000:3.0:1789760784.603312:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714693227985 time 714707970518 state 0x2 ref 0 00000100:00100000:1.0:1789760784.610382:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760784.610539:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844480 00000100:00100000:0.0:1789760784.610558:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844480 00000100:00100000:0.0:1789760784.610621:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714693227985 time 714707970518 state 0x0 ref 1 00000100:00100000:0.0:1789760784.610636:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714708230934 time 714707970518 state 0x1 ref 1 00000100:00100000:0.0:1789760784.610762:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714708230934 time 714735367546 state 0x0 ref 1 00000100:00100000:0.0:1789760784.610777:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760784.610830:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78577664, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x25798 00010000:00010000:0.0:1789760784.610839:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.610859:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760784.610864:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.610870:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.610881:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.616210:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986491 is committed 00002000:00010000:2.0:1789760784.616316:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78577664 -> 78581760 00002000:00010000:2.0:1789760784.616321:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153496 -> 153504 00010000:00010000:2.0:1789760784.616334:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.616347:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.616357:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.616364:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.616368:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.616377:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.616465:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.616475:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1499 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78577664->78581759] (req 78577664->78581759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.616781:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6001us (6403us total) trans 4294986491 rc 0/0 00000100:00100000:3.0:1789760784.616791:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19195 00000100:00100000:3.0:1789760784.616801:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714708230934 time 714735367546 state 0x2 ref 0 00000100:00100000:3.0:1789760784.629309:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760784.630042:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844608 00000100:00100000:0.0:1789760784.630052:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844608 00000100:00100000:1.0:1789760784.630234:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714708230934 time 714735367546 state 0x0 ref 1 00000100:00100000:1.0:1789760784.630249:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714735627962 time 714735367546 state 0x1 ref 1 00000100:00100000:1.0:1789760784.630325:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714735627962 time 714754930678 state 0x0 ref 1 00000100:00100000:1.0:1789760784.630338:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503844608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.630385:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78581760, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257a0 00010000:00010000:1.0:1789760784.630409:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.630424:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.630429:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.630435:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.630448:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.636023:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986492 is committed 00002000:00010000:1.0:1789760784.636148:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78581760 -> 78585856 00002000:00010000:1.0:1789760784.636154:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153504 -> 153512 00010000:00010000:1.0:1789760784.636167:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.636179:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.636188:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.636196:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.636201:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.636210:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.636229:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.636243:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78581760->78585855] (req 78581760->78585855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.636484:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503844608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6144us (7179us total) trans 4294986492 rc 0/0 00000100:00100000:1.0:1789760784.636495:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19196 00000100:00100000:1.0:1789760784.636504:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714735627962 time 714754930678 state 0x2 ref 0 00000100:00100000:3.0:1789760784.644292:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.644510:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844736 00000100:00100000:2.0:1789760784.644521:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844736 00000100:00100000:2.0:1789760784.644692:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714735627962 time 714754930678 state 0x0 ref 1 00000100:00100000:2.0:1789760784.644709:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714755191094 time 714754930678 state 0x1 ref 1 00000100:00100000:2.0:1789760784.644780:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714755191094 time 714769384968 state 0x0 ref 1 00000100:00100000:2.0:1789760784.644794:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.644845:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78585856, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257a8 00010000:00010000:2.0:1789760784.644856:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.644871:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.644875:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.644881:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.644895:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.650504:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986493 is committed 00002000:00010000:2.0:1789760784.650743:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78585856 -> 78589952 00002000:00010000:2.0:1789760784.650748:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153512 -> 153520 00010000:00010000:2.0:1789760784.650763:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.650775:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.650790:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.650797:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.650802:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.650811:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.650831:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.650838:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78585856->78589951] (req 78585856->78589951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.651287:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6490us (6999us total) trans 4294986493 rc 0/0 00000100:00100000:2.0:1789760784.651299:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19197 00000100:00100000:2.0:1789760784.651309:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714755191094 time 714769384968 state 0x2 ref 0 00000100:00100000:1.0:1789760784.663868:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.663916:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844864 00000100:00100000:3.0:1789760784.663927:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844864 00000100:00100000:3.0:1789760784.663983:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714755191094 time 714769384968 state 0x0 ref 1 00000100:00100000:3.0:1789760784.663998:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714769645384 time 714769384968 state 0x1 ref 1 00000100:00100000:3.0:1789760784.664028:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714769645384 time 714788634449 state 0x0 ref 1 00000100:00100000:3.0:1789760784.664041:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.664088:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78589952, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257b0 00010000:00010000:3.0:1789760784.664097:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.664218:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.664223:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.664230:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.664244:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.668931:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986494 is committed 00002000:00010000:2.0:1789760784.669123:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78589952 -> 78594048 00002000:00010000:2.0:1789760784.669128:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153520 -> 153528 00010000:00010000:2.0:1789760784.669142:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.669252:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.669262:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.669278:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.669283:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.669319:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.669342:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.669349:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f14ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78589952->78594047] (req 78589952->78594047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.669605:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5553us (5735us total) trans 4294986494 rc 0/0 00000100:00100000:2.0:1789760784.669617:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19198 00000100:00100000:2.0:1789760784.669627:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714769645384 time 714788634449 state 0x2 ref 0 00000100:00100000:3.0:1789760784.673467:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.673566:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503844992 00000100:00100000:1.0:1789760784.673577:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503844992 00000100:00100000:1.0:1789760784.673647:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714769645384 time 714788634449 state 0x0 ref 1 00000100:00100000:1.0:1789760784.673676:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714788894865 time 714788634449 state 0x1 ref 1 00000100:00100000:1.0:1789760784.673700:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714788894865 time 714798307240 state 0x0 ref 1 00000100:00100000:1.0:1789760784.673712:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.673754:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78594048, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257b8 00010000:00010000:1.0:1789760784.673764:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.673778:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.673783:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.673790:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.673802:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.680107:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986495 is committed 00002000:00010000:3.0:1789760784.680245:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78594048 -> 78598144 00002000:00010000:3.0:1789760784.680250:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153528 -> 153536 00010000:00010000:3.0:1789760784.680264:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.680407:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.680417:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.680425:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.680430:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.680439:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.680461:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.680474:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f14b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78594048->78598143] (req 78594048->78598143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.680590:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503844992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6875us (7127us total) trans 4294986495 rc 0/0 00000100:00100000:3.0:1789760784.680601:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19199 00000100:00100000:3.0:1789760784.680611:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714788894865 time 714798307240 state 0x2 ref 0 00000100:00100000:0.0:1789760784.685936:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.686011:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503845120 00000100:00100000:1.0:1789760784.686021:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503845120 00000100:00100000:1.0:1789760784.686079:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714788894865 time 714798307240 state 0x0 ref 1 00000100:00100000:1.0:1789760784.686094:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714798567656 time 714798307240 state 0x1 ref 1 00000100:00100000:1.0:1789760784.686127:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714798567656 time 714810733530 state 0x0 ref 1 00000100:00100000:1.0:1789760784.686139:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503845120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.686189:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78598144, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257c0 00010000:00010000:1.0:1789760784.686199:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.686214:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.686219:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.686225:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.686239:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.691026:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986496 is committed 00002000:00010000:2.0:1789760784.691253:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78598144 -> 78602240 00002000:00010000:2.0:1789760784.691259:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153536 -> 153544 00010000:00010000:2.0:1789760784.691327:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.691339:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.691349:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.691356:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.691360:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.691369:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.691484:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.691491:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f14bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78598144->78602239] (req 78598144->78602239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.691590:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503845120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5448us (5659us total) trans 4294986496 rc 0/0 00000100:00100000:2.0:1789760784.691720:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19200 00000100:00100000:2.0:1789760784.691731:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714798567656 time 714810733530 state 0x2 ref 0 00000100:00100000:1.0:1789760784.695012:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.695089:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503845248 00000100:00100000:2.0:1789760784.695099:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503845248 00000100:00100000:2.0:1789760784.695162:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714798567656 time 714810733530 state 0x0 ref 1 00000100:00100000:2.0:1789760784.695177:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714810993946 time 714810733530 state 0x1 ref 1 00000100:00100000:2.0:1789760784.695207:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714810993946 time 714819813449 state 0x0 ref 1 00000100:00100000:2.0:1789760784.695219:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503845248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.695268:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78602240, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257c8 00010000:00010000:2.0:1789760784.695289:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.695306:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.695311:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.695317:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.695331:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.700327:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986497 is committed 00002000:00010000:2.0:1789760784.701053:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78602240 -> 78606336 00002000:00010000:2.0:1789760784.701060:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153544 -> 153552 00010000:00010000:2.0:1789760784.701073:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.701092:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.701101:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.701109:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.701114:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.701124:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.701144:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.701152:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78602240->78606335] (req 78602240->78606335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.703443:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503845248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8214us (8428us total) trans 4294986497 rc 0/0 00000100:00100000:2.0:1789760784.703458:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19201 00000100:00100000:2.0:1789760784.703468:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714810993946 time 714819813449 state 0x2 ref 0 00000100:00100000:3.0:1789760784.711069:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.714792:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503845376 00000100:00100000:2.0:1789760784.714809:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503845376 00000100:00100000:2.0:1789760784.714874:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714810993946 time 714819813449 state 0x0 ref 1 00000100:00100000:2.0:1789760784.714891:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714820073865 time 714819813449 state 0x1 ref 1 00000100:00100000:2.0:1789760784.714937:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714820073865 time 714839543846 state 0x0 ref 1 00000100:00100000:2.0:1789760784.714951:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503845376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.715006:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78606336, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257d0 00010000:00010000:2.0:1789760784.715017:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.715032:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.715036:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.715043:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.715056:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.723411:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986498 is committed 00002000:00010000:2.0:1789760784.723500:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78606336 -> 78610432 00002000:00010000:2.0:1789760784.723505:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153552 -> 153560 00010000:00010000:2.0:1789760784.723518:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.723530:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.723552:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.723560:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.723565:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.723574:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.723704:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.723714:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78606336->78610431] (req 78606336->78610431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.723842:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503845376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8889us (12778us total) trans 4294986498 rc 0/0 00000100:00100000:2.0:1789760784.723852:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19202 00000100:00100000:2.0:1789760784.723862:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714820073865 time 714839543846 state 0x2 ref 0 00000100:00100000:1.0:1789760784.726788:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.726893:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503845504 00000100:00100000:3.0:1789760784.726903:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503845504 00000100:00100000:3.0:1789760784.726958:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714820073865 time 714839543846 state 0x0 ref 1 00000100:00100000:3.0:1789760784.726973:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714839804262 time 714839543846 state 0x1 ref 1 00000100:00100000:3.0:1789760784.727005:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714839804262 time 714851611151 state 0x0 ref 1 00000100:00100000:3.0:1789760784.727016:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503845504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.727061:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78610432, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257d8 00010000:00010000:3.0:1789760784.727070:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.727085:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.727090:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.727096:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.727122:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.738019:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986499 is committed 00002000:00010000:3.0:1789760784.738204:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78610432 -> 78614528 00002000:00010000:3.0:1789760784.738209:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153560 -> 153568 00010000:00010000:3.0:1789760784.738223:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.738237:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.738245:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.738253:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.738257:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.738319:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.738342:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.738351:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78610432->78614527] (req 78610432->78614527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.738853:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503845504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11832us (12069us total) trans 4294986499 rc 0/0 00000100:00100000:3.0:1789760784.738866:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19203 00000100:00100000:3.0:1789760784.738875:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714839804262 time 714851611151 state 0x2 ref 0 00000100:00100000:1.0:1789760784.744028:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.744125:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503845632 00000100:00100000:2.0:1789760784.744136:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503845632 00000100:00100000:2.0:1789760784.744191:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714839804262 time 714851611151 state 0x0 ref 1 00000100:00100000:2.0:1789760784.744206:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714851871567 time 714851611151 state 0x1 ref 1 00000100:00100000:2.0:1789760784.744237:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714851871567 time 714868844285 state 0x0 ref 1 00000100:00100000:2.0:1789760784.744250:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503845632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.744305:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78614528, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257e0 00010000:00010000:2.0:1789760784.744314:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.744329:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.744333:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.744339:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.744354:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.754221:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986500 is committed 00002000:00010000:2.0:1789760784.754381:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78614528 -> 78618624 00002000:00010000:2.0:1789760784.754386:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153568 -> 153576 00010000:00010000:2.0:1789760784.754418:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.754430:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.754437:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.754448:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.754453:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.754461:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.754479:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.754486:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78614528->78618623] (req 78614528->78618623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.754642:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503845632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10382us (10611us total) trans 4294986500 rc 0/0 00000100:00100000:2.0:1789760784.754823:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19204 00000100:00100000:2.0:1789760784.754835:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714851871567 time 714868844285 state 0x2 ref 0 00000100:00100000:0.0:1789760784.768066:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.768563:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503845760 00000100:00100000:2.0:1789760784.768575:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503845760 00000100:00100000:2.0:1789760784.768700:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714851871567 time 714868844285 state 0x0 ref 1 00000100:00100000:2.0:1789760784.768716:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714869104701 time 714868844285 state 0x1 ref 1 00000100:00100000:2.0:1789760784.768790:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714869104701 time 714893395276 state 0x0 ref 1 00000100:00100000:2.0:1789760784.768806:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503845760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.768887:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78618624, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257e8 00010000:00010000:2.0:1789760784.768896:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.768912:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.768917:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.768923:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.768937:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.774834:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986501 is committed 00002000:00010000:2.0:1789760784.774988:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78618624 -> 78622720 00002000:00010000:2.0:1789760784.774993:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153576 -> 153584 00010000:00010000:2.0:1789760784.775005:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.775018:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.775026:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.775033:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.775037:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.775045:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.775064:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.775072:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f14df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78618624->78622719] (req 78618624->78622719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.775281:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503845760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6465us (7212us total) trans 4294986501 rc 0/0 00000100:00100000:2.0:1789760784.775291:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19205 00000100:00100000:2.0:1789760784.775300:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714869104701 time 714893395276 state 0x2 ref 0 00000100:00100000:1.0:1789760784.786889:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.787741:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503845888 00000100:00100000:3.0:1789760784.787753:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503845888 00000100:00100000:3.0:1789760784.787814:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714869104701 time 714893395276 state 0x0 ref 1 00000100:00100000:3.0:1789760784.787833:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714893655692 time 714893395276 state 0x1 ref 1 00000100:00100000:3.0:1789760784.787868:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714893655692 time 714912474179 state 0x0 ref 1 00000100:00100000:3.0:1789760784.787881:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503845888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.787930:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78622720, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x257f0 00010000:00010000:3.0:1789760784.787940:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.787955:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.787960:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.787965:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.787978:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.798031:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986502 is committed 00002000:00010000:2.0:1789760784.798197:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78622720 -> 78626816 00002000:00010000:2.0:1789760784.798202:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:2.0:1789760784.798205:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00002000:00010000:2.0:1789760784.798207:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153584 -> 153592 00010000:00010000:2.0:1789760784.798220:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.798232:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.798245:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.798253:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.798258:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.798326:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.798347:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.798354:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f14e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78622720->78626815] (req 78622720->78626815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.798522:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503845888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10638us (11640us total) trans 4294986502 rc 0/0 00000100:00100000:2.0:1789760784.798533:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19206 00000100:00100000:2.0:1789760784.798541:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714893655692 time 714912474179 state 0x2 ref 0 00000100:00100000:1.0:1789760784.801324:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.801533:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846016 00000100:00100000:3.0:1789760784.801543:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846016 00000100:00100000:1.0:1789760784.801730:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714893655692 time 714912474179 state 0x0 ref 1 00000100:00100000:1.0:1789760784.801746:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714912734595 time 714912474179 state 0x1 ref 1 00000100:00100000:1.0:1789760784.801808:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714912734595 time 714926413507 state 0x0 ref 1 00000100:00100000:1.0:1789760784.801820:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.801867:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78626816, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x257f8 00010000:00010000:1.0:1789760784.801877:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.801891:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760784.801895:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.801902:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.801921:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.820385:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986503 is committed 00002000:00010000:1.0:1789760784.820531:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78626816 -> 78630912 00002000:00010000:1.0:1789760784.820536:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153592 -> 153600 00010000:00010000:1.0:1789760784.820560:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.820574:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.820710:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.820721:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.820725:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.820735:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.820756:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.820765:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f14ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78626816->78630911] (req 78626816->78630911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.820919:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19095us (19656us total) trans 4294986503 rc 0/0 00000100:00100000:1.0:1789760784.820931:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19207 00000100:00100000:1.0:1789760784.820942:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714912734595 time 714926413507 state 0x2 ref 0 00000100:00100000:1.0:1789760784.842867:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.843377:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846144 00000100:00100000:2.0:1789760784.843387:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846144 00000100:00100000:2.0:1789760784.843731:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714912734595 time 714926413507 state 0x0 ref 1 00000100:00100000:2.0:1789760784.843750:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714926673923 time 714926413507 state 0x1 ref 1 00000100:00100000:2.0:1789760784.843783:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714926673923 time 714968390439 state 0x0 ref 1 00000100:00100000:2.0:1789760784.843797:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.843845:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78630912, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25800 00010000:00010000:2.0:1789760784.843855:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.843869:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760784.843874:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.843880:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.843893:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.858451:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986504 is committed 00002000:00010000:2.0:1789760784.858573:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78630912 -> 78635008 00002000:00010000:2.0:1789760784.858578:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153600 -> 153608 00010000:00010000:2.0:1789760784.858591:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.858603:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.858612:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.858623:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.858628:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.858637:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.858666:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.858674:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f14f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78630912->78635007] (req 78630912->78635007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.858764:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14966us (15903us total) trans 4294986504 rc 0/0 00000100:00100000:2.0:1789760784.858775:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19208 00000100:00100000:2.0:1789760784.858784:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714926673923 time 714968390439 state 0x2 ref 0 00000100:00100000:2.0:1789760784.861918:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.862002:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846272 00000100:00100000:3.0:1789760784.862011:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846272 00000100:00100000:3.0:1789760784.862066:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714926673923 time 714968390439 state 0x0 ref 1 00000100:00100000:3.0:1789760784.862081:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714968650855 time 714968390439 state 0x1 ref 1 00000100:00100000:3.0:1789760784.862114:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714968650855 time 714986720057 state 0x0 ref 1 00000100:00100000:3.0:1789760784.862136:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ece300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.862187:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78635008, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25808 00010000:00010000:3.0:1789760784.862197:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.862212:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760784.862216:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.862222:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.862245:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.875833:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986505 is committed 00002000:00010000:3.0:1789760784.875975:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78635008 -> 78639104 00002000:00010000:3.0:1789760784.875981:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153608 -> 153616 00010000:00010000:3.0:1789760784.875995:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.876008:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.876016:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.876022:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.876027:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.876036:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.876055:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.876062:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f14fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78635008->78639103] (req 78635008->78639103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.876166:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ece300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14036us (14252us total) trans 4294986505 rc 0/0 00000100:00100000:3.0:1789760784.876176:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19209 00000100:00100000:3.0:1789760784.876185:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714968650855 time 714986720057 state 0x2 ref 0 00000100:00100000:1.0:1789760784.880569:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.880665:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846400 00000100:00100000:2.0:1789760784.880675:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846400 00000100:00100000:2.0:1789760784.880728:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714968650855 time 714986720057 state 0x0 ref 1 00000100:00100000:2.0:1789760784.880743:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714986980473 time 714986720057 state 0x1 ref 1 00000100:00100000:2.0:1789760784.880775:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714986980473 time 715005381931 state 0x0 ref 1 00000100:00100000:2.0:1789760784.880788:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.880841:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78639104, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25810 00010000:00010000:2.0:1789760784.880850:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.880866:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760784.880871:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.880877:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.880893:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.892668:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986506 is committed 00002000:00010000:2.0:1789760784.894174:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78639104 -> 78643200 00002000:00010000:2.0:1789760784.894182:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153616 -> 153624 00010000:00010000:2.0:1789760784.894196:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.894209:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.894218:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.894226:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.894231:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.894581:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.894604:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.894612:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1502 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78639104->78643199] (req 78639104->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.894761:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13969us (14196us total) trans 4294986506 rc 0/0 00000100:00100000:2.0:1789760784.894771:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19210 00000100:00100000:2.0:1789760784.894780:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714986980473 time 715005381931 state 0x2 ref 0 00000100:00100000:1.0:1789760784.898081:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.898165:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846528 00000100:00100000:2.0:1789760784.898176:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846528 00000100:00100000:2.0:1789760784.898228:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 714986980473 time 715005381931 state 0x0 ref 1 00000100:00100000:2.0:1789760784.898253:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715005642347 time 715005381931 state 0x1 ref 1 00000100:00100000:2.0:1789760784.898287:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715005642347 time 715022893637 state 0x0 ref 1 00000100:00100000:2.0:1789760784.898300:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.898347:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78643200, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25818 00010000:00010000:2.0:1789760784.898357:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.898372:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760784.898377:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.898383:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.898427:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.905570:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986507 is committed 00002000:00010000:2.0:1789760784.906539:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78643200 -> 78647296 00002000:00010000:2.0:1789760784.906553:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153624 -> 153632 00010000:00010000:2.0:1789760784.906566:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.906579:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.906588:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.906595:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.906599:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.906609:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.906628:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.906637:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1509 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78643200->78647295] (req 78643200->78647295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.906740:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8438us (8664us total) trans 4294986507 rc 0/0 00000100:00100000:2.0:1789760784.906750:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19211 00000100:00100000:2.0:1789760784.906759:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715005642347 time 715022893637 state 0x2 ref 0 00000100:00100000:1.0:1789760784.909740:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.909825:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846656 00000100:00100000:2.0:1789760784.909834:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846656 00000100:00100000:2.0:1789760784.909885:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715005642347 time 715022893637 state 0x0 ref 1 00000100:00100000:2.0:1789760784.909900:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715023154053 time 715022893637 state 0x1 ref 1 00000100:00100000:2.0:1789760784.909928:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715023154053 time 715034534977 state 0x0 ref 1 00000100:00100000:2.0:1789760784.909940:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.909984:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78647296, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25820 00010000:00010000:2.0:1789760784.909993:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.910007:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760784.910012:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.910019:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.910030:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.914770:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760784.916469:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986508 is committed 00002000:00010000:1.0:1789760784.916642:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78647296 -> 78651392 00002000:00010000:1.0:1789760784.916657:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153632 -> 153640 00010000:00010000:1.0:1789760784.916671:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.916684:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.916693:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.916700:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.916704:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.916713:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.916735:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.916742:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1510 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78647296->78651391] (req 78647296->78651391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.916845:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6901us (7108us total) trans 4294986508 rc 0/0 00000100:00100000:1.0:1789760784.916855:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19212 00000100:00100000:1.0:1789760784.916865:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715023154053 time 715034534977 state 0x2 ref 0 00000100:00100000:1.0:1789760784.919884:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.919965:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846784 00000100:00100000:3.0:1789760784.919976:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846784 00000100:00100000:3.0:1789760784.920030:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715023154053 time 715034534977 state 0x0 ref 1 00000100:00100000:3.0:1789760784.920044:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715034795393 time 715034534977 state 0x1 ref 1 00000100:00100000:3.0:1789760784.920075:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715034795393 time 715044681054 state 0x0 ref 1 00000100:00100000:3.0:1789760784.920087:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.920132:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78651392, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25828 00010000:00010000:3.0:1789760784.920142:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.920157:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760784.920162:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.920168:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.920182:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.928650:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986509 is committed 00002000:00010000:3.0:1789760784.928878:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78651392 -> 78655488 00002000:00010000:3.0:1789760784.928883:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153640 -> 153648 00010000:00010000:3.0:1789760784.928955:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.928974:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.928983:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.928990:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.928995:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.929004:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.929024:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.929032:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1517 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78651392->78655487] (req 78651392->78655487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.930788:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503846784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10692us (10901us total) trans 4294986509 rc 0/0 00000100:00100000:3.0:1789760784.930802:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19213 00000100:00100000:3.0:1789760784.930812:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715034795393 time 715044681054 state 0x2 ref 0 00000100:00100000:2.0:1789760784.935775:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.935869:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503846912 00000100:00100000:1.0:1789760784.935881:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503846912 00000100:00100000:1.0:1789760784.935997:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715034795393 time 715044681054 state 0x0 ref 1 00000100:00100000:1.0:1789760784.936011:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715044941470 time 715044681054 state 0x1 ref 1 00000100:00100000:1.0:1789760784.936183:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715044941470 time 715060787863 state 0x0 ref 1 00000100:00100000:1.0:1789760784.936198:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.936243:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78655488, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25830 00010000:00010000:1.0:1789760784.936253:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936269:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760784.936274:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.936281:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.936296:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.942180:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986510 is committed 00002000:00010000:1.0:1789760784.945208:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78655488 -> 78659584 00002000:00010000:1.0:1789760784.945216:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153648 -> 153656 00010000:00010000:1.0:1789760784.945227:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945237:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945243:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945248:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760784.945251:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945257:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945323:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945329:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f151e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78655488->78659583] (req 78655488->78659583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.945438:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503846912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9241us (9670us total) trans 4294986510 rc 0/0 00000100:00100000:1.0:1789760784.945447:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19214 00000100:00100000:1.0:1789760784.945454:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715044941470 time 715060787863 state 0x2 ref 0 00000100:00100000:2.0:1789760784.951289:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.951422:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847040 00000100:00100000:1.0:1789760784.951432:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847040 00000100:00100000:1.0:1789760784.951485:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715044941470 time 715060787863 state 0x0 ref 1 00000100:00100000:1.0:1789760784.951499:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715061048279 time 715060787863 state 0x1 ref 1 00000100:00100000:1.0:1789760784.951572:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715061048279 time 715076178291 state 0x0 ref 1 00000100:00100000:1.0:1789760784.951586:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.951640:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78659584, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25838 00010000:00010000:1.0:1789760784.951658:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.951674:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760784.951679:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.951685:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.951699:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.958839:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986511 is committed 00002000:00010000:2.0:1789760784.958941:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78659584 -> 78663680 00002000:00010000:2.0:1789760784.958946:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153656 -> 153664 00010000:00010000:2.0:1789760784.958958:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.958967:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.958973:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.958977:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.958980:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.958987:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.959005:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.959010:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1525 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78659584->78663679] (req 78659584->78663679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.959217:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7629us (7941us total) trans 4294986511 rc 0/0 00000100:00100000:2.0:1789760784.959226:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19215 00000100:00100000:2.0:1789760784.959235:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715061048279 time 715076178291 state 0x2 ref 0 00000100:00100000:2.0:1789760784.964027:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760784.966448:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847168 00000100:00100000:1.0:1789760784.966469:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847168 00000100:00100000:1.0:1789760784.966531:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715061048279 time 715076178291 state 0x0 ref 1 00000100:00100000:1.0:1789760784.966556:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715076438707 time 715076178291 state 0x1 ref 1 00000100:00100000:1.0:1789760784.966597:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715076438707 time 715091204227 state 0x0 ref 1 00000100:00100000:1.0:1789760784.966610:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760784.966824:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78663680, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25840 00010000:00010000:1.0:1789760784.966838:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.966854:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760784.966858:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.966865:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.966879:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.976122:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986512 is committed 00002000:00010000:2.0:1789760784.976829:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78663680 -> 78667776 00002000:00010000:2.0:1789760784.976835:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153664 -> 153672 00010000:00010000:2.0:1789760784.976850:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.976862:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.976871:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.976878:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760784.976882:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.976893:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.976910:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.976917:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f152c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78663680->78667775] (req 78663680->78667775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.977069:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10456us (13047us total) trans 4294986512 rc 0/0 00000100:00100000:2.0:1789760784.977079:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19216 00000100:00100000:2.0:1789760784.977088:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715076438707 time 715091204227 state 0x2 ref 0 00000100:00100000:1.0:1789760784.981756:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760784.981964:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847296 00000100:00100000:3.0:1789760784.981974:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847296 00000100:00100000:3.0:1789760784.982086:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715076438707 time 715091204227 state 0x0 ref 1 00000100:00100000:3.0:1789760784.982101:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715091464643 time 715091204227 state 0x1 ref 1 00000100:00100000:3.0:1789760784.982168:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715091464643 time 715106774354 state 0x0 ref 1 00000100:00100000:3.0:1789760784.982181:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503847296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760784.982224:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78667776, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25848 00010000:00010000:3.0:1789760784.982233:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.982248:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760784.982253:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.982259:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.982273:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.987845:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986513 is committed 00002000:00010000:3.0:1789760784.989821:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78667776 -> 78671872 00002000:00010000:3.0:1789760784.989830:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153672 -> 153680 00010000:00010000:3.0:1789760784.989844:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.989858:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.989872:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.989880:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760784.989884:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.989893:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.989911:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.989919:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1533 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78667776->78671871] (req 78667776->78671871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.990027:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503847296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7843us (8275us total) trans 4294986513 rc 0/0 00000100:00100000:3.0:1789760784.990038:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19217 00000100:00100000:3.0:1789760784.990047:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715091464643 time 715106774354 state 0x2 ref 0 00000100:00100000:1.0:1789760784.998175:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760784.998349:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847424 00000100:00100000:2.0:1789760784.998359:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847424 00000100:00100000:2.0:1789760784.998460:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715091464643 time 715106774354 state 0x0 ref 1 00000100:00100000:2.0:1789760784.998659:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715107034770 time 715106774354 state 0x1 ref 1 00000100:00100000:2.0:1789760784.998730:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715107034770 time 715123335286 state 0x0 ref 1 00000100:00100000:2.0:1789760784.998743:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc14450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760784.998846:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78671872, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25850 00010000:00010000:2.0:1789760784.998853:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.998867:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760784.998870:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.998874:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.998884:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.003876:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986514 is committed 00002000:00010000:2.0:1789760785.004002:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78671872 -> 78675968 00002000:00010000:2.0:1789760785.004007:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153680 -> 153688 00010000:00010000:2.0:1789760785.004020:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.004033:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.004041:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.004049:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.004055:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.004064:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.004089:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.004097:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f153a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78671872->78675967] (req 78671872->78675967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.004224:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc14450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5478us (6053us total) trans 4294986514 rc 0/0 00000100:00100000:2.0:1789760785.004234:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19218 00000100:00100000:2.0:1789760785.004244:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715107034770 time 715123335286 state 0x2 ref 0 00000100:00100000:2.0:1789760785.006843:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.006923:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847552 00000100:00100000:1.0:1789760785.006932:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847552 00000100:00100000:1.0:1789760785.006983:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715107034770 time 715123335286 state 0x0 ref 1 00000100:00100000:1.0:1789760785.006998:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715123595702 time 715123335286 state 0x1 ref 1 00000100:00100000:1.0:1789760785.007026:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715123595702 time 715131632695 state 0x0 ref 1 00000100:00100000:1.0:1789760785.007037:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.007081:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78675968, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25858 00010000:00010000:1.0:1789760785.007090:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.007105:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.007109:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.007115:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.007128:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.011251:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986515 is committed 00002000:00010000:2.0:1789760785.011716:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78675968 -> 78680064 00002000:00010000:2.0:1789760785.011722:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153688 -> 153696 00010000:00010000:2.0:1789760785.011735:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.011748:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.011758:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.011766:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.011771:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.011781:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.011802:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.011810:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1541 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78675968->78680063] (req 78675968->78680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.011961:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4861us (5062us total) trans 4294986515 rc 0/0 00000100:00100000:2.0:1789760785.011972:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19219 00000100:00100000:2.0:1789760785.011982:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715123595702 time 715131632695 state 0x2 ref 0 00000100:00100000:1.0:1789760785.015024:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.015246:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847680 00000100:00100000:2.0:1789760785.015257:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847680 00000100:00100000:2.0:1789760785.015319:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715123595702 time 715131632695 state 0x0 ref 1 00000100:00100000:2.0:1789760785.015334:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715131893111 time 715131632695 state 0x1 ref 1 00000100:00100000:2.0:1789760785.015366:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715131893111 time 715139972613 state 0x0 ref 1 00000100:00100000:2.0:1789760785.015378:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.015453:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78680064, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25860 00010000:00010000:2.0:1789760785.015461:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.015490:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.015495:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.015502:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.015515:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.019135:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986516 is committed 00002000:00010000:2.0:1789760785.019239:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78680064 -> 78684160 00002000:00010000:2.0:1789760785.019244:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153696 -> 153704 00010000:00010000:2.0:1789760785.019258:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.019280:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.019289:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.019297:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.019301:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.019310:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.019333:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.019340:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1548 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78680064->78684159] (req 78680064->78684159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.019469:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4088us (4450us total) trans 4294986516 rc 0/0 00000100:00100000:2.0:1789760785.019479:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19220 00000100:00100000:2.0:1789760785.019489:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715131893111 time 715139972613 state 0x2 ref 0 00000100:00100000:1.0:1789760785.025787:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.025909:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847808 00000100:00100000:3.0:1789760785.025921:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847808 00000100:00100000:3.0:1789760785.025984:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715131893111 time 715139972613 state 0x0 ref 1 00000100:00100000:3.0:1789760785.025999:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715140233029 time 715139972613 state 0x1 ref 1 00000100:00100000:3.0:1789760785.026028:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715140233029 time 715150634822 state 0x0 ref 1 00000100:00100000:3.0:1789760785.026040:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.026091:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78684160, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25868 00010000:00010000:3.0:1789760785.026102:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.026121:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760785.026127:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.026133:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.026149:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.031428:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986517 is committed 00002000:00010000:2.0:1789760785.031559:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78684160 -> 78688256 00002000:00010000:2.0:1789760785.031566:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153704 -> 153712 00010000:00010000:2.0:1789760785.031580:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.031593:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.031601:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.031609:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.031613:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.031629:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.031649:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.031657:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f154f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78684160->78688255] (req 78684160->78688255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.031752:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5709us (5973us total) trans 4294986517 rc 0/0 00000100:00100000:2.0:1789760785.031763:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19221 00000100:00100000:2.0:1789760785.031772:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715140233029 time 715150634822 state 0x2 ref 0 00000100:00100000:1.0:1789760785.042743:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.042849:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503847936 00000100:00100000:2.0:1789760785.042859:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503847936 00000100:00100000:2.0:1789760785.042912:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715140233029 time 715150634822 state 0x0 ref 1 00000100:00100000:2.0:1789760785.042926:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715150895238 time 715150634822 state 0x1 ref 1 00000100:00100000:2.0:1789760785.042954:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715150895238 time 715167561032 state 0x0 ref 1 00000100:00100000:2.0:1789760785.042966:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.043010:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78688256, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25870 00010000:00010000:2.0:1789760785.043020:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.043037:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.043042:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.043048:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.043061:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.055045:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986518 is committed 00002000:00010000:2.0:1789760785.055471:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78688256 -> 78692352 00002000:00010000:2.0:1789760785.055476:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153712 -> 153720 00010000:00010000:2.0:1789760785.055489:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.055502:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.055511:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.055519:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.055524:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.055532:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.055733:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.055742:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1556 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78688256->78692351] (req 78688256->78692351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.055866:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503847936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12897us (13127us total) trans 4294986518 rc 0/0 00000100:00100000:2.0:1789760785.055876:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19222 00000100:00100000:2.0:1789760785.055886:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715150895238 time 715167561032 state 0x2 ref 0 00000100:00100000:1.0:1789760785.064526:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.064876:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848064 00000100:00100000:2.0:1789760785.064886:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848064 00000100:00100000:2.0:1789760785.064939:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715150895238 time 715167561032 state 0x0 ref 1 00000100:00100000:2.0:1789760785.064953:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715167821448 time 715167561032 state 0x1 ref 1 00000100:00100000:2.0:1789760785.064989:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715167821448 time 715189594711 state 0x0 ref 1 00000100:00100000:2.0:1789760785.065002:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.065051:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78692352, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25878 00010000:00010000:2.0:1789760785.065059:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.065075:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.065080:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.065086:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.065100:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.073880:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986519 is committed 00002000:00010000:1.0:1789760785.074449:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78692352 -> 78696448 00002000:00010000:1.0:1789760785.074454:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153720 -> 153728 00010000:00010000:1.0:1789760785.074467:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.074481:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.074490:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.074498:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.074503:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.074513:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.074538:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.074556:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f155d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78692352->78696447] (req 78692352->78696447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.074650:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9647us (10129us total) trans 4294986519 rc 0/0 00000100:00100000:1.0:1789760785.074661:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19223 00000100:00100000:1.0:1789760785.074670:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715167821448 time 715189594711 state 0x2 ref 0 00000100:00100000:0.0:1789760785.079740:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.080102:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848192 00000100:00100000:1.0:1789760785.080112:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848192 00000100:00100000:1.0:1789760785.080168:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715167821448 time 715189594711 state 0x0 ref 1 00000100:00100000:1.0:1789760785.080185:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715189855127 time 715189594711 state 0x1 ref 1 00000100:00100000:1.0:1789760785.080217:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715189855127 time 715204823778 state 0x0 ref 1 00000100:00100000:1.0:1789760785.080230:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503848192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.080287:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78696448, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25880 00010000:00010000:1.0:1789760785.080296:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.080312:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.080317:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.080322:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.080335:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.086596:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986520 is committed 00002000:00010000:1.0:1789760785.086760:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78696448 -> 78700544 00002000:00010000:1.0:1789760785.086766:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153728 -> 153736 00010000:00010000:1.0:1789760785.086780:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086794:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086803:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086810:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.086815:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086825:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086846:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086861:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1564 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78696448->78700543] (req 78696448->78700543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.086969:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503848192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6735us (7234us total) trans 4294986520 rc 0/0 00000100:00100000:1.0:1789760785.086980:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19224 00000100:00100000:1.0:1789760785.087001:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715189855127 time 715204823778 state 0x2 ref 0 00000100:00100000:2.0:1789760785.090246:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.090336:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848320 00000100:00100000:3.0:1789760785.090347:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848320 00000100:00100000:3.0:1789760785.090442:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715189855127 time 715204823778 state 0x0 ref 1 00000100:00100000:3.0:1789760785.090458:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715205084194 time 715204823778 state 0x1 ref 1 00000100:00100000:3.0:1789760785.090482:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715205084194 time 715215089314 state 0x0 ref 1 00000100:00100000:3.0:1789760785.090494:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.090552:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78700544, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25888 00010000:00010000:3.0:1789760785.090562:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.090577:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760785.090777:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.090785:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.090800:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.096819:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986521 is committed 00002000:00010000:2.0:1789760785.097061:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78700544 -> 78704640 00002000:00010000:2.0:1789760785.097067:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153736 -> 153744 00010000:00010000:2.0:1789760785.097079:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.097092:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.097112:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.097120:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.097125:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.097135:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.097161:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.097170:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f156b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78700544->78704639] (req 78700544->78704639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.097368:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6870us (7126us total) trans 4294986521 rc 0/0 00000100:00100000:2.0:1789760785.097379:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19225 00000100:00100000:2.0:1789760785.097429:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715205084194 time 715215089314 state 0x2 ref 0 00000100:00100000:2.0:1789760785.107049:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.114087:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848448 00000100:00100000:1.0:1789760785.114101:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848448 00000100:00100000:1.0:1789760785.114166:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715205084194 time 715215089314 state 0x0 ref 1 00000100:00100000:1.0:1789760785.114182:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715215349730 time 715215089314 state 0x1 ref 1 00000100:00100000:1.0:1789760785.114225:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715215349730 time 715238832184 state 0x0 ref 1 00000100:00100000:1.0:1789760785.114240:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503848448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.114698:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78704640, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25890 00010000:00010000:1.0:1789760785.114708:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.114725:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.114729:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.114735:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.114753:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.120314:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986522 is committed 00002000:00010000:1.0:1789760785.129842:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78704640 -> 78708736 00002000:00010000:1.0:1789760785.129852:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153744 -> 153752 00010000:00010000:1.0:1789760785.129866:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.129880:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.129889:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.129897:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.129915:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.129925:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.129945:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.129953:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1572 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78704640->78708735] (req 78704640->78708735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.130054:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503848448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15813us (23010us total) trans 4294986522 rc 0/0 00000100:00100000:1.0:1789760785.130065:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19226 00000100:00100000:1.0:1789760785.130074:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715215349730 time 715238832184 state 0x2 ref 0 00000100:00100000:2.0:1789760785.132984:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.133071:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848576 00000100:00100000:1.0:1789760785.133081:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848576 00000100:00100000:1.0:1789760785.133138:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715215349730 time 715238832184 state 0x0 ref 1 00000100:00100000:1.0:1789760785.133154:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715239092600 time 715238832184 state 0x1 ref 1 00000100:00100000:1.0:1789760785.133182:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715239092600 time 715257788620 state 0x0 ref 1 00000100:00100000:1.0:1789760785.133194:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.133246:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78708736, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25898 00010000:00010000:1.0:1789760785.133254:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.133270:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.133275:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.133281:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.133295:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.139496:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986523 is committed 00002000:00010000:1.0:1789760785.139609:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78708736 -> 78712832 00002000:00010000:1.0:1789760785.139615:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153752 -> 153760 00010000:00010000:1.0:1789760785.139627:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.139641:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.139650:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.139658:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.139663:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.139672:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.139691:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.139698:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1579 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78708736->78712831] (req 78708736->78712831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.139804:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6603us (6820us total) trans 4294986523 rc 0/0 00000100:00100000:1.0:1789760785.139817:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19227 00000100:00100000:1.0:1789760785.139828:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715239092600 time 715257788620 state 0x2 ref 0 00000100:00100000:2.0:1789760785.147709:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.147817:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848704 00000100:00100000:1.0:1789760785.147828:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848704 00000100:00100000:1.0:1789760785.147880:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715239092600 time 715257788620 state 0x0 ref 1 00000100:00100000:1.0:1789760785.147894:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715258049036 time 715257788620 state 0x1 ref 1 00000100:00100000:1.0:1789760785.148068:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715258049036 time 715272673410 state 0x0 ref 1 00000100:00100000:1.0:1789760785.148082:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.148127:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78712832, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258a0 00010000:00010000:1.0:1789760785.148137:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.148152:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.148157:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.148163:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.148177:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.155634:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986524 is committed 00002000:00010000:1.0:1789760785.155755:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78712832 -> 78716928 00002000:00010000:1.0:1789760785.155760:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153760 -> 153768 00010000:00010000:1.0:1789760785.155774:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.155787:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.155797:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.155804:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.155809:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.155817:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.155837:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.155846:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1580 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78712832->78716927] (req 78712832->78716927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.155938:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7854us (8557us total) trans 4294986524 rc 0/0 00000100:00100000:1.0:1789760785.155949:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19228 00000100:00100000:1.0:1789760785.155958:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715258049036 time 715272673410 state 0x2 ref 0 00000100:00100000:2.0:1789760785.164370:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.164578:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848832 00000100:00100000:3.0:1789760785.164597:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848832 00000100:00100000:3.0:1789760785.164659:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715258049036 time 715272673410 state 0x0 ref 1 00000100:00100000:3.0:1789760785.164675:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715272933826 time 715272673410 state 0x1 ref 1 00000100:00100000:3.0:1789760785.164706:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715272933826 time 715289312367 state 0x0 ref 1 00000100:00100000:3.0:1789760785.164718:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.164769:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78716928, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258a8 00010000:00010000:3.0:1789760785.164778:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.164799:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760785.164804:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.164811:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.164824:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.172939:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986525 is committed 00002000:00010000:2.0:1789760785.173085:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78716928 -> 78721024 00002000:00010000:2.0:1789760785.173091:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153768 -> 153776 00010000:00010000:2.0:1789760785.173106:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.173119:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.173127:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.173139:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.173144:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.173154:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.173174:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.173181:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1587 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78716928->78721023] (req 78716928->78721023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.173596:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503848832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8874us (9229us total) trans 4294986525 rc 0/0 00000100:00100000:2.0:1789760785.173608:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19229 00000100:00100000:2.0:1789760785.173618:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715272933826 time 715289312367 state 0x2 ref 0 00000100:00100000:2.0:1789760785.177959:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.178073:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503848960 00000100:00100000:1.0:1789760785.178084:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503848960 00000100:00100000:1.0:1789760785.178140:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715272933826 time 715289312367 state 0x0 ref 1 00000100:00100000:1.0:1789760785.178155:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715289572783 time 715289312367 state 0x1 ref 1 00000100:00100000:1.0:1789760785.178214:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715289572783 time 715302819966 state 0x0 ref 1 00000100:00100000:1.0:1789760785.178361:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503848960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.178481:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78721024, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258b0 00010000:00010000:1.0:1789760785.178491:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.178506:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.178511:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.178517:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.178529:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.183386:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986526 is committed 00002000:00010000:1.0:1789760785.183537:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78721024 -> 78725120 00002000:00010000:1.0:1789760785.183554:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153776 -> 153784 00010000:00010000:1.0:1789760785.183568:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.183586:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.183595:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.183604:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.183608:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.183617:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.183638:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.183646:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f158e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78721024->78725119] (req 78721024->78725119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.183749:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503848960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5519us (5794us total) trans 4294986526 rc 0/0 00000100:00100000:1.0:1789760785.183759:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19230 00000100:00100000:1.0:1789760785.183768:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715289572783 time 715302819966 state 0x2 ref 0 00000100:00100000:1.0:1789760785.187799:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.187983:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849088 00000100:00100000:2.0:1789760785.187993:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849088 00000100:00100000:2.0:1789760785.188050:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715289572783 time 715302819966 state 0x0 ref 1 00000100:00100000:2.0:1789760785.188066:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715303080382 time 715302819966 state 0x1 ref 1 00000100:00100000:2.0:1789760785.188105:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715303080382 time 715312711090 state 0x0 ref 1 00000100:00100000:2.0:1789760785.188119:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.188167:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78725120, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258b8 00010000:00010000:2.0:1789760785.188175:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.188191:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.188196:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.188203:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.188217:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.198906:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986527 is committed 00002000:00010000:2.0:1789760785.199059:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78725120 -> 78729216 00002000:00010000:2.0:1789760785.199064:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153784 -> 153792 00010000:00010000:2.0:1789760785.199077:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.199089:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.199097:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.199104:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.199108:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.199116:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.199135:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.199143:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1595 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78725120->78729215] (req 78725120->78729215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.199239:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11118us (11445us total) trans 4294986527 rc 0/0 00000100:00100000:2.0:1789760785.199250:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19231 00000100:00100000:2.0:1789760785.199260:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715303080382 time 715312711090 state 0x2 ref 0 00000100:00100000:2.0:1789760785.211115:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.211966:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849216 00000100:00100000:1.0:1789760785.211978:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849216 00000100:00100000:1.0:1789760785.212037:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715303080382 time 715312711090 state 0x0 ref 1 00000100:00100000:1.0:1789760785.212052:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715312971506 time 715312711090 state 0x1 ref 1 00000100:00100000:1.0:1789760785.212090:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715312971506 time 715336696255 state 0x0 ref 1 00000100:00100000:1.0:1789760785.212104:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.212154:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78729216, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258c0 00010000:00010000:1.0:1789760785.212163:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.212179:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.212184:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.212190:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.212204:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.223330:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986528 is committed 00002000:00010000:1.0:1789760785.223502:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78729216 -> 78733312 00002000:00010000:1.0:1789760785.223507:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153792 -> 153800 00010000:00010000:1.0:1789760785.223521:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.223535:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.223554:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.223562:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.223571:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.223581:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.223601:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.223609:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f159c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78729216->78733311] (req 78729216->78733311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.223757:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11650us (12645us total) trans 4294986528 rc 0/0 00000100:00100000:1.0:1789760785.223768:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19232 00000100:00100000:1.0:1789760785.223777:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715312971506 time 715336696255 state 0x2 ref 0 00000100:00100000:0.0:1789760785.227449:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.227656:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849344 00000100:00100000:2.0:1789760785.227667:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849344 00000100:00100000:2.0:1789760785.227723:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715312971506 time 715336696255 state 0x0 ref 1 00000100:00100000:2.0:1789760785.227739:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715336956671 time 715336696255 state 0x1 ref 1 00000100:00100000:2.0:1789760785.227800:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715336956671 time 715352405372 state 0x0 ref 1 00000100:00100000:2.0:1789760785.227813:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.227864:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78733312, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258c8 00010000:00010000:2.0:1789760785.227874:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.227890:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.227894:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.227901:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.227915:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.237511:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986529 is committed 00002000:00010000:0.0:1789760785.237890:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78733312 -> 78737408 00002000:00010000:0.0:1789760785.237896:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153800 -> 153808 00010000:00010000:0.0:1789760785.237910:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.237923:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.237932:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.237939:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760785.237944:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.237952:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.238048:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.238056:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78733312->78737407] (req 78733312->78737407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.238209:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10393us (10941us total) trans 4294986529 rc 0/0 00000100:00100000:0.0:1789760785.238220:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19233 00000100:00100000:0.0:1789760785.238230:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715336956671 time 715352405372 state 0x2 ref 0 00000100:00100000:2.0:1789760785.246262:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.246377:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849472 00000100:00100000:1.0:1789760785.246429:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849472 00000100:00100000:1.0:1789760785.246489:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715336956671 time 715352405372 state 0x0 ref 1 00000100:00100000:1.0:1789760785.246505:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715352665788 time 715352405372 state 0x1 ref 1 00000100:00100000:1.0:1789760785.246539:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715352665788 time 715371145139 state 0x0 ref 1 00000100:00100000:1.0:1789760785.246562:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503849472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.246619:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78737408, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258d0 00010000:00010000:1.0:1789760785.246628:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.246645:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.246650:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.246655:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.246669:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.257102:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986530 is committed 00002000:00010000:1.0:1789760785.257351:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78737408 -> 78741504 00002000:00010000:1.0:1789760785.257357:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153808 -> 153816 00010000:00010000:1.0:1789760785.257369:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.257381:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.257486:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.257499:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.257504:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.257513:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.257532:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.257551:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78737408->78741503] (req 78737408->78741503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.257843:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503849472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11270us (11578us total) trans 4294986530 rc 0/0 00000100:00100000:0.0:1789760785.257856:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19234 00000100:00100000:0.0:1789760785.257866:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715352665788 time 715371145139 state 0x2 ref 0 00000100:00100000:0.0:1789760785.261168:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.261686:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849600 00000100:00100000:1.0:1789760785.261696:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849600 00000100:00100000:1.0:1789760785.261750:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715352665788 time 715371145139 state 0x0 ref 1 00000100:00100000:1.0:1789760785.261766:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715371405555 time 715371145139 state 0x1 ref 1 00000100:00100000:1.0:1789760785.261787:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715371405555 time 715386394526 state 0x0 ref 1 00000100:00100000:1.0:1789760785.261799:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503849600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.262009:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78741504, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258d8 00010000:00010000:1.0:1789760785.262019:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.262041:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.262045:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.262051:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.262064:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.283488:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986531 is committed 00002000:00010000:2.0:1789760785.283709:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78741504 -> 78745600 00002000:00010000:2.0:1789760785.283714:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153816 -> 153824 00010000:00010000:2.0:1789760785.283728:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.283741:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.283750:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.283757:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.283762:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.283771:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.283793:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.283801:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f15b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78741504->78745599] (req 78741504->78745599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.290901:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503849600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 29092us (29730us total) trans 4294986531 rc 0/0 00000100:00100000:3.0:1789760785.290916:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19235 00000100:00100000:3.0:1789760785.290927:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715371405555 time 715386394526 state 0x2 ref 0 00000100:00100000:2.0:1789760785.295334:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.295462:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849728 00000100:00100000:1.0:1789760785.295472:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849728 00000100:00100000:1.0:1789760785.295529:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715371405555 time 715386394526 state 0x0 ref 1 00000100:00100000:1.0:1789760785.295557:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715386654942 time 715386394526 state 0x1 ref 1 00000100:00100000:1.0:1789760785.295598:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715386654942 time 715420204907 state 0x0 ref 1 00000100:00100000:1.0:1789760785.295611:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.295659:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78745600, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258e0 00010000:00010000:1.0:1789760785.295669:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.295686:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.295691:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.295698:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.295717:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.303124:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986532 is committed 00002000:00010000:1.0:1789760785.303410:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78745600 -> 78749696 00002000:00010000:1.0:1789760785.303415:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153824 -> 153832 00010000:00010000:1.0:1789760785.303429:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.303442:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.303451:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.303458:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.303463:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.303472:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.303491:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.303499:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78745600->78749695] (req 78745600->78749695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.303606:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503849728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7992us (8277us total) trans 4294986532 rc 0/0 00000100:00100000:1.0:1789760785.303617:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19236 00000100:00100000:1.0:1789760785.303635:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715386654942 time 715420204907 state 0x2 ref 0 00000100:00100000:3.0:1789760785.315421:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760785.315533:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849856 00000100:00100000:0.0:1789760785.315554:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849856 00000100:00100000:0.0:1789760785.315612:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715386654942 time 715420204907 state 0x0 ref 1 00000100:00100000:0.0:1789760785.315691:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715420465323 time 715420204907 state 0x1 ref 1 00000100:00100000:0.0:1789760785.315759:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715420465323 time 715440364398 state 0x0 ref 1 00000100:00100000:0.0:1789760785.315774:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503849856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760785.315824:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78749696, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258e8 00010000:00010000:0.0:1789760785.315833:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.315850:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760785.315855:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.315863:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.315878:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.323492:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986533 is committed 00002000:00010000:1.0:1789760785.323582:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78749696 -> 78753792 00002000:00010000:1.0:1789760785.323587:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153832 -> 153840 00010000:00010000:1.0:1789760785.323604:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.323617:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.323735:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.323743:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.323748:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.323757:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.323777:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.323791:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f15bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78749696->78753791] (req 78749696->78753791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.323884:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503849856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8109us (8467us total) trans 4294986533 rc 0/0 00000100:00100000:1.0:1789760785.323893:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19237 00000100:00100000:1.0:1789760785.323903:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715420465323 time 715440364398 state 0x2 ref 0 00000100:00100000:0.0:1789760785.327784:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.327937:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503849984 00000100:00100000:2.0:1789760785.327948:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503849984 00000100:00100000:2.0:1789760785.328003:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715420465323 time 715440364398 state 0x0 ref 1 00000100:00100000:2.0:1789760785.328018:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715440624814 time 715440364398 state 0x1 ref 1 00000100:00100000:2.0:1789760785.328052:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715440624814 time 715452657666 state 0x0 ref 1 00000100:00100000:2.0:1789760785.328065:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503849984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.328113:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78753792, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258f0 00010000:00010000:2.0:1789760785.328122:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.328143:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.328147:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.328154:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.328168:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.336248:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986534 is committed 00002000:00010000:1.0:1789760785.336429:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78753792 -> 78757888 00002000:00010000:1.0:1789760785.336435:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153840 -> 153848 00010000:00010000:1.0:1789760785.336449:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.336461:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.336470:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.336478:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.336482:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.336493:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.336517:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.336525:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f15c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78753792->78757887] (req 78753792->78757887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.336730:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503849984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8662us (8951us total) trans 4294986534 rc 0/0 00000100:00100000:1.0:1789760785.336741:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19238 00000100:00100000:1.0:1789760785.336751:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715440624814 time 715452657666 state 0x2 ref 0 00000100:00100000:0.0:1789760785.340576:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.340714:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503850112 00000100:00100000:3.0:1789760785.340725:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503850112 00000100:00100000:3.0:1789760785.340782:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715440624814 time 715452657666 state 0x0 ref 1 00000100:00100000:3.0:1789760785.340797:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715452918082 time 715452657666 state 0x1 ref 1 00000100:00100000:3.0:1789760785.340826:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715452918082 time 715465433396 state 0x0 ref 1 00000100:00100000:3.0:1789760785.340839:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.340883:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78757888, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x258f8 00010000:00010000:3.0:1789760785.340891:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.340906:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760785.340910:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.340916:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.340929:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.345816:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986535 is committed 00002000:00010000:0.0:1789760785.345916:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78757888 -> 78761984 00002000:00010000:0.0:1789760785.345921:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153848 -> 153856 00010000:00010000:0.0:1789760785.345935:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.345945:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.345951:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.345956:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760785.345959:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.345965:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.345980:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.345986:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f15cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78757888->78761983] (req 78757888->78761983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.346073:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5233us (5503us total) trans 4294986535 rc 0/0 00000100:00100000:0.0:1789760785.346081:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19239 00000100:00100000:0.0:1789760785.346088:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715452918082 time 715465433396 state 0x2 ref 0 00000100:00100000:3.0:1789760785.353753:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.354425:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503850240 00000100:00100000:1.0:1789760785.354437:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503850240 00000100:00100000:1.0:1789760785.354494:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715452918082 time 715465433396 state 0x0 ref 1 00000100:00100000:1.0:1789760785.354509:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715465693812 time 715465433396 state 0x1 ref 1 00000100:00100000:1.0:1789760785.354560:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715465693812 time 715479166036 state 0x0 ref 1 00000100:00100000:1.0:1789760785.354574:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503850240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.354945:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78761984, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25900 00010000:00010000:1.0:1789760785.354956:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.354971:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.354976:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.354982:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.354996:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.361414:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986536 is committed 00002000:00010000:1.0:1789760785.361656:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78761984 -> 78766080 00002000:00010000:1.0:1789760785.361662:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153856 -> 153864 00010000:00010000:1.0:1789760785.361675:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.361687:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.361695:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.361702:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.361707:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.361716:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.361736:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.361749:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78761984->78766079] (req 78761984->78766079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.361884:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503850240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7307us (8140us total) trans 4294986536 rc 0/0 00000100:00100000:1.0:1789760785.361894:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19240 00000100:00100000:1.0:1789760785.361903:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715465693812 time 715479166036 state 0x2 ref 0 00000100:00100000:3.0:1789760785.368163:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.368263:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503850368 00000100:00100000:1.0:1789760785.368281:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503850368 00000100:00100000:1.0:1789760785.368335:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715465693812 time 715479166036 state 0x0 ref 1 00000100:00100000:1.0:1789760785.368351:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715479426452 time 715479166036 state 0x1 ref 1 00000100:00100000:1.0:1789760785.368382:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715479426452 time 715492988008 state 0x0 ref 1 00000100:00100000:1.0:1789760785.368429:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.368475:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78766080, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25908 00010000:00010000:1.0:1789760785.368484:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.368499:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.368504:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.368510:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.368528:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.374019:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986537 is committed 00002000:00010000:1.0:1789760785.374224:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78766080 -> 78770176 00002000:00010000:1.0:1789760785.374229:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153864 -> 153872 00010000:00010000:1.0:1789760785.374243:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.374255:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.374593:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.374602:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.374607:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.374617:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.374636:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.374644:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f15db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78766080->78770175] (req 78766080->78770175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.374737:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6306us (6578us total) trans 4294986537 rc 0/0 00000100:00100000:1.0:1789760785.374747:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19241 00000100:00100000:1.0:1789760785.374757:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715479426452 time 715492988008 state 0x2 ref 0 00000100:00100000:0.0:1789760785.379860:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.379906:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503850496 00000100:00100000:1.0:1789760785.379916:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503850496 00000100:00100000:1.0:1789760785.379969:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715479426452 time 715492988008 state 0x0 ref 1 00000100:00100000:1.0:1789760785.379985:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715493248424 time 715492988008 state 0x1 ref 1 00000100:00100000:1.0:1789760785.380022:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715493248424 time 715504628030 state 0x0 ref 1 00000100:00100000:1.0:1789760785.380035:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.380083:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78770176, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25910 00010000:00010000:1.0:1789760785.380093:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.380106:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.380110:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.380114:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.380124:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.386297:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986538 is committed 00002000:00010000:1.0:1789760785.386571:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78770176 -> 78774272 00002000:00010000:1.0:1789760785.386695:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153872 -> 153880 00010000:00010000:1.0:1789760785.386711:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.386726:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.386736:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.386744:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.386749:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.386759:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.386781:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.386788:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78770176->78774271] (req 78770176->78774271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.386884:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6847us (7030us total) trans 4294986538 rc 0/0 00000100:00100000:1.0:1789760785.386895:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19242 00000100:00100000:1.0:1789760785.386914:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715493248424 time 715504628030 state 0x2 ref 0 00000100:00100000:3.0:1789760785.390245:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760785.390491:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503850624 00000100:00100000:0.0:1789760785.390502:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503850624 00000100:00100000:0.0:1789760785.390567:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715493248424 time 715504628030 state 0x0 ref 1 00000100:00100000:0.0:1789760785.390582:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715504888446 time 715504628030 state 0x1 ref 1 00000100:00100000:0.0:1789760785.390709:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715504888446 time 715515314834 state 0x0 ref 1 00000100:00100000:0.0:1789760785.390723:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503850624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760785.390771:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78774272, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25918 00010000:00010000:0.0:1789760785.390781:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.390802:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760785.390806:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.390812:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.390825:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.395627:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986539 is committed 00002000:00010000:0.0:1789760785.402440:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78774272 -> 78778368 00002000:00010000:0.0:1789760785.402449:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153880 -> 153888 00010000:00010000:0.0:1789760785.402463:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.402474:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.402482:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.402489:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760785.402493:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.402502:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.402525:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.402531:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f15e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78774272->78778367] (req 78774272->78778367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.402643:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503850624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11919us (12404us total) trans 4294986539 rc 0/0 00000100:00100000:0.0:1789760785.402653:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19243 00000100:00100000:0.0:1789760785.403212:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715504888446 time 715515314834 state 0x2 ref 0 00000100:00100000:3.0:1789760785.406634:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.406923:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503850752 00000100:00100000:2.0:1789760785.406935:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503850752 00000100:00100000:2.0:1789760785.406990:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715504888446 time 715515314834 state 0x0 ref 1 00000100:00100000:2.0:1789760785.407005:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715515575250 time 715515314834 state 0x1 ref 1 00000100:00100000:2.0:1789760785.407041:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715515575250 time 715531647161 state 0x0 ref 1 00000100:00100000:2.0:1789760785.407052:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.407099:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78778368, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25920 00010000:00010000:2.0:1789760785.407108:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.407124:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.407128:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.407135:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.407149:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.412990:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986540 is committed 00002000:00010000:1.0:1789760785.413610:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78778368 -> 78782464 00002000:00010000:1.0:1789760785.413616:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153888 -> 153896 00010000:00010000:1.0:1789760785.413629:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.413642:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.413652:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.413659:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.413713:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.413723:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.413742:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.413749:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f15f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78778368->78782463] (req 78778368->78782463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.413899:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503850752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6843us (7268us total) trans 4294986540 rc 0/0 00000100:00100000:1.0:1789760785.413909:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19244 00000100:00100000:1.0:1789760785.413918:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715515575250 time 715531647161 state 0x2 ref 0 00000100:00100000:2.0:1789760785.417907:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.418051:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503850880 00000100:00100000:1.0:1789760785.418060:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503850880 00000100:00100000:1.0:1789760785.418112:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715515575250 time 715531647161 state 0x0 ref 1 00000100:00100000:1.0:1789760785.418127:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715531907577 time 715531647161 state 0x1 ref 1 00000100:00100000:1.0:1789760785.418158:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715531907577 time 715542764884 state 0x0 ref 1 00000100:00100000:1.0:1789760785.418169:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503850880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.418230:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78782464, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25928 00010000:00010000:1.0:1789760785.418239:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.418255:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.418259:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.418272:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.418286:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.429777:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986541 is committed 00002000:00010000:2.0:1789760785.430674:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78782464 -> 78786560 00002000:00010000:2.0:1789760785.430681:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153896 -> 153904 00010000:00010000:2.0:1789760785.430695:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.430706:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.430716:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.430723:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.430728:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.430737:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.430759:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.430773:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f15f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78782464->78786559] (req 78782464->78786559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.431022:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503850880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12848us (13118us total) trans 4294986541 rc 0/0 00000100:00100000:2.0:1789760785.431033:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19245 00000100:00100000:2.0:1789760785.431043:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715531907577 time 715542764884 state 0x2 ref 0 00000100:00100000:3.0:1789760785.434120:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760785.434224:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851008 00000100:00100000:0.0:1789760785.434235:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851008 00000100:00100000:0.0:1789760785.434429:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715531907577 time 715542764884 state 0x0 ref 1 00000100:00100000:0.0:1789760785.434445:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715543025300 time 715542764884 state 0x1 ref 1 00000100:00100000:0.0:1789760785.434473:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715543025300 time 715559078490 state 0x0 ref 1 00000100:00100000:3.0:1789760785.434546:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503851008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.434717:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78786560, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25930 00010000:00010000:3.0:1789760785.434727:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.434783:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760785.435091:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.435098:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.435113:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.443918:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986542 is committed 00002000:00010000:3.0:1789760785.444034:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78786560 -> 78790656 00002000:00010000:3.0:1789760785.444039:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153904 -> 153912 00010000:00010000:3.0:1789760785.444051:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.444064:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.444073:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.444081:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760785.444086:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.444095:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.444114:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.444122:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f15fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78786560->78790655] (req 78786560->78790655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.444373:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503851008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9884us (10257us total) trans 4294986542 rc 0/0 00000100:00100000:3.0:1789760785.444385:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19246 00000100:00100000:3.0:1789760785.444428:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715543025300 time 715559078490 state 0x2 ref 0 00000100:00100000:3.0:1789760785.455554:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760785.455772:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851136 00000100:00100000:0.0:1789760785.455782:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851136 00000100:00100000:0.0:1789760785.455835:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715543025300 time 715559078490 state 0x0 ref 1 00000100:00100000:0.0:1789760785.455851:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715559338906 time 715559078490 state 0x1 ref 1 00000100:00100000:0.0:1789760785.455887:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715559338906 time 715580493296 state 0x0 ref 1 00000100:00100000:0.0:1789760785.455899:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503851136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760785.455944:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78790656, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25938 00010000:00010000:0.0:1789760785.455953:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.455970:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760785.455974:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.455981:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.455994:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.463200:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986543 is committed 00002000:00010000:1.0:1789760785.463564:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78790656 -> 78794752 00002000:00010000:1.0:1789760785.463569:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153912 -> 153920 00010000:00010000:1.0:1789760785.463650:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.463663:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.463672:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.463679:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.463684:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.463693:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.463712:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.463720:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1605 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78790656->78794751] (req 78790656->78794751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.463855:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503851136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7953us (8315us total) trans 4294986543 rc 0/0 00000100:00100000:1.0:1789760785.463866:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19247 00000100:00100000:1.0:1789760785.463874:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715559338906 time 715580493296 state 0x2 ref 0 00000100:00100000:2.0:1789760785.472289:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.472425:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851264 00000100:00100000:1.0:1789760785.472435:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851264 00000100:00100000:1.0:1789760785.472493:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715559338906 time 715580493296 state 0x0 ref 1 00000100:00100000:1.0:1789760785.472510:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715580753712 time 715580493296 state 0x1 ref 1 00000100:00100000:1.0:1789760785.472550:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715580753712 time 715597147106 state 0x0 ref 1 00000100:00100000:1.0:1789760785.472563:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503851264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.472613:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78794752, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25940 00010000:00010000:1.0:1789760785.472622:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.472637:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.472642:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.472648:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.472661:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.477884:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986544 is committed 00002000:00010000:1.0:1789760785.478350:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78794752 -> 78798848 00002000:00010000:1.0:1789760785.478356:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153920 -> 153928 00010000:00010000:1.0:1789760785.478370:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.478383:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.478430:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.478438:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.478442:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.478452:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.478472:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.478481:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f160c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78794752->78798847] (req 78794752->78798847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.478742:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503851264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6176us (6457us total) trans 4294986544 rc 0/0 00000100:00100000:1.0:1789760785.478752:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19248 00000100:00100000:1.0:1789760785.478764:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715580753712 time 715597147106 state 0x2 ref 0 00000100:00100000:3.0:1789760785.482117:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.483008:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851392 00000100:00100000:2.0:1789760785.483019:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851392 00000100:00100000:2.0:1789760785.483076:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715580753712 time 715597147106 state 0x0 ref 1 00000100:00100000:2.0:1789760785.483092:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715597407522 time 715597147106 state 0x1 ref 1 00000100:00100000:2.0:1789760785.483126:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715597407522 time 715607732546 state 0x0 ref 1 00000100:00100000:2.0:1789760785.483141:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503851392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.483190:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78798848, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25948 00010000:00010000:2.0:1789760785.483199:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.483215:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.483220:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.483226:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.483238:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.490525:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986545 is committed 00002000:00010000:2.0:1789760785.490646:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78798848 -> 78802944 00002000:00010000:2.0:1789760785.490651:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153928 -> 153936 00010000:00010000:2.0:1789760785.490664:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.490677:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.490687:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.490694:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.490698:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.490708:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.490728:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.490742:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1613 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78798848->78802943] (req 78798848->78802943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.490836:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503851392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7695us (8724us total) trans 4294986545 rc 0/0 00000100:00100000:2.0:1789760785.490847:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19249 00000100:00100000:2.0:1789760785.490858:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715597407522 time 715607732546 state 0x2 ref 0 00000100:00100000:3.0:1789760785.495848:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.496089:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851520 00000100:00100000:1.0:1789760785.496099:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851520 00000100:00100000:1.0:1789760785.496154:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715597407522 time 715607732546 state 0x0 ref 1 00000100:00100000:1.0:1789760785.496170:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715607992962 time 715607732546 state 0x1 ref 1 00000100:00100000:1.0:1789760785.496201:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715607992962 time 715620807075 state 0x0 ref 1 00000100:00100000:1.0:1789760785.496213:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503851520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.496259:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78802944, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25950 00010000:00010000:1.0:1789760785.496268:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.496284:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.496288:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.496295:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.496309:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.503339:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986546 is committed 00002000:00010000:1.0:1789760785.503498:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78802944 -> 78807040 00002000:00010000:1.0:1789760785.503504:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153936 -> 153944 00010000:00010000:1.0:1789760785.503518:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.503536:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.503554:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.503562:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.503566:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.503576:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.503604:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.503613:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f161a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78802944->78807039] (req 78802944->78807039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.503707:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503851520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7491us (7865us total) trans 4294986546 rc 0/0 00000100:00100000:1.0:1789760785.503718:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19250 00000100:00100000:1.0:1789760785.503729:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715607992962 time 715620807075 state 0x2 ref 0 00000100:00100000:0.0:1789760785.512500:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.512609:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851648 00000100:00100000:1.0:1789760785.512620:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851648 00000100:00100000:1.0:1789760785.512677:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715607992962 time 715620807075 state 0x0 ref 1 00000100:00100000:1.0:1789760785.512692:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715621067491 time 715620807075 state 0x1 ref 1 00000100:00100000:1.0:1789760785.512726:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715621067491 time 715637332435 state 0x0 ref 1 00000100:00100000:1.0:1789760785.512739:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503851648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.512785:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78807040, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25958 00010000:00010000:1.0:1789760785.512794:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.512810:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.512814:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.512821:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.512835:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.519228:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986547 is committed 00002000:00010000:1.0:1789760785.519370:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78807040 -> 78811136 00002000:00010000:1.0:1789760785.519375:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153944 -> 153952 00010000:00010000:1.0:1789760785.519403:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.519416:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.519431:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.519439:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.519444:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.519454:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.519473:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.519481:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1621 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78807040->78811135] (req 78807040->78811135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.519641:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503851648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6899us (7145us total) trans 4294986547 rc 0/0 00000100:00100000:1.0:1789760785.519651:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19251 00000100:00100000:1.0:1789760785.519662:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715621067491 time 715637332435 state 0x2 ref 0 00000100:00100000:3.0:1789760785.525712:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.526384:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851776 00000100:00100000:2.0:1789760785.526471:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851776 00000100:00100000:2.0:1789760785.526529:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715621067491 time 715637332435 state 0x0 ref 1 00000100:00100000:2.0:1789760785.526553:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715637592851 time 715637332435 state 0x1 ref 1 00000100:00100000:2.0:1789760785.526631:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715637592851 time 715651185880 state 0x0 ref 1 00000100:00100000:2.0:1789760785.526646:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503851776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.526698:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78811136, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25960 00010000:00010000:2.0:1789760785.526707:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.526725:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.526729:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.526735:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.526754:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.532678:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986548 is committed 00002000:00010000:2.0:1789760785.532839:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78811136 -> 78815232 00002000:00010000:2.0:1789760785.532845:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153952 -> 153960 00010000:00010000:2.0:1789760785.532859:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.532872:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.532881:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.532888:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.532893:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.533131:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.533150:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.533158:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1628 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78811136->78815231] (req 78811136->78815231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.533288:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503851776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6640us (7580us total) trans 4294986548 rc 0/0 00000100:00100000:2.0:1789760785.533298:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19252 00000100:00100000:2.0:1789760785.533308:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715637592851 time 715651185880 state 0x2 ref 0 00000100:00100000:3.0:1789760785.536712:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.536788:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503851904 00000100:00100000:2.0:1789760785.536798:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503851904 00000100:00100000:2.0:1789760785.536854:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715637592851 time 715651185880 state 0x0 ref 1 00000100:00100000:2.0:1789760785.536870:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715651446296 time 715651185880 state 0x1 ref 1 00000100:00100000:2.0:1789760785.536911:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715651446296 time 715661510950 state 0x0 ref 1 00000100:00100000:2.0:1789760785.536925:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503851904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.536983:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78815232, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25968 00010000:00010000:2.0:1789760785.536992:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.537008:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.537013:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.537019:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.537032:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.541842:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986549 is committed 00002000:00010000:2.0:1789760785.542056:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78815232 -> 78819328 00002000:00010000:2.0:1789760785.542062:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153960 -> 153968 00010000:00010000:2.0:1789760785.542074:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.542087:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.542096:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.542103:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.542108:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.542117:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.542136:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.542145:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f162f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78815232->78819327] (req 78815232->78819327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.542279:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503851904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5351us (5572us total) trans 4294986549 rc 0/0 00000100:00100000:2.0:1789760785.542290:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19253 00000100:00100000:2.0:1789760785.542300:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715651446296 time 715661510950 state 0x2 ref 0 00000100:00100000:0.0:1789760785.546947:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.547123:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852032 00000100:00100000:1.0:1789760785.547133:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852032 00000100:00100000:1.0:1789760785.547186:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715651446296 time 715661510950 state 0x0 ref 1 00000100:00100000:1.0:1789760785.547202:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715661771366 time 715661510950 state 0x1 ref 1 00000100:00100000:1.0:1789760785.547278:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715661771366 time 715671884224 state 0x0 ref 1 00000100:00100000:1.0:1789760785.547293:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.547340:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78819328, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25970 00010000:00010000:1.0:1789760785.547349:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.547364:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.547368:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.547375:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.547419:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.554914:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986550 is committed 00002000:00010000:1.0:1789760785.555083:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78819328 -> 78823424 00002000:00010000:1.0:1789760785.555088:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153968 -> 153976 00010000:00010000:1.0:1789760785.555102:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.555114:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.555122:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.555135:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.555141:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.555150:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.555168:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.555176:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1636 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78819328->78823423] (req 78819328->78823423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.555868:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8573us (8926us total) trans 4294986550 rc 0/0 00000100:00100000:1.0:1789760785.555879:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19254 00000100:00100000:1.0:1789760785.555889:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715661771366 time 715671884224 state 0x2 ref 0 00000100:00100000:3.0:1789760785.560068:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.560344:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852160 00000100:00100000:1.0:1789760785.560353:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852160 00000100:00100000:1.0:1789760785.560886:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715661771366 time 715671884224 state 0x0 ref 1 00000100:00100000:1.0:1789760785.560912:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715672144640 time 715671884224 state 0x1 ref 1 00000100:00100000:1.0:1789760785.560944:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715672144640 time 715685550630 state 0x0 ref 1 00000100:00100000:1.0:1789760785.560962:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.561022:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78823424, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25978 00010000:00010000:1.0:1789760785.561031:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.561047:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.561052:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.561058:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.561072:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.569229:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986551 is committed 00002000:00010000:0.0:1789760785.573163:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78823424 -> 78827520 00002000:00010000:0.0:1789760785.573173:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153976 -> 153984 00010000:00010000:0.0:1789760785.573186:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.573205:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.573214:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.573221:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760785.573225:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.573233:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.573254:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.573262:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f163d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78823424->78827519] (req 78823424->78827519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.573533:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12569us (13470us total) trans 4294986551 rc 0/0 00000100:00100000:0.0:1789760785.573552:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19255 00000100:00100000:0.0:1789760785.573562:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715672144640 time 715685550630 state 0x2 ref 0 00000100:00100000:3.0:1789760785.577749:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.577848:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852288 00000100:00100000:2.0:1789760785.577859:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852288 00000100:00100000:2.0:1789760785.578047:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715672144640 time 715685550630 state 0x0 ref 1 00000100:00100000:2.0:1789760785.578064:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715685811046 time 715685550630 state 0x1 ref 1 00000100:00100000:2.0:1789760785.578135:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715685811046 time 715702740762 state 0x0 ref 1 00000100:00100000:2.0:1789760785.578148:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503852288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.578213:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78827520, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25980 00010000:00010000:2.0:1789760785.578222:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.578238:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.578243:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.578249:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.578273:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.582876:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986552 is committed 00002000:00010000:2.0:1789760785.583136:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78827520 -> 78831616 00002000:00010000:2.0:1789760785.583141:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153984 -> 153992 00010000:00010000:2.0:1789760785.583154:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583166:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583175:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583183:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.583187:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583195:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583215:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.583222:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1644 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78827520->78831615] (req 78827520->78831615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.583364:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503852288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5213us (5620us total) trans 4294986552 rc 0/0 00000100:00100000:2.0:1789760785.583374:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19256 00000100:00100000:2.0:1789760785.583385:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715685811046 time 715702740762 state 0x2 ref 0 00000100:00100000:2.0:1789760785.587223:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.587384:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852416 00000100:00100000:1.0:1789760785.587461:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852416 00000100:00100000:1.0:1789760785.587515:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715685811046 time 715702740762 state 0x0 ref 1 00000100:00100000:1.0:1789760785.587531:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715703001178 time 715702740762 state 0x1 ref 1 00000100:00100000:1.0:1789760785.587565:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715703001178 time 715712170827 state 0x0 ref 1 00000100:00100000:1.0:1789760785.587577:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.587624:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78831616, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25988 00010000:00010000:1.0:1789760785.587634:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.587769:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.587774:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.587780:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.587795:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.596830:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986553 is committed 00002000:00010000:1.0:1789760785.596928:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78831616 -> 78835712 00002000:00010000:1.0:1789760785.596933:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 153992 -> 154000 00010000:00010000:1.0:1789760785.596945:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.596958:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.596968:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.596976:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.596981:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.596990:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597015:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597022:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f164b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78831616->78835711] (req 78831616->78835711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.597128:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9547us (9910us total) trans 4294986553 rc 0/0 00000100:00100000:1.0:1789760785.597137:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19257 00000100:00100000:1.0:1789760785.597146:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715703001178 time 715712170827 state 0x2 ref 0 00000100:00100000:3.0:1789760785.605165:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.606096:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852544 00000100:00100000:1.0:1789760785.606107:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852544 00000100:00100000:2.0:1789760785.606442:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715703001178 time 715712170827 state 0x0 ref 1 00000100:00100000:2.0:1789760785.606457:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715712431243 time 715712170827 state 0x1 ref 1 00000100:00100000:2.0:1789760785.606487:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715712431243 time 715731093219 state 0x0 ref 1 00000100:00100000:2.0:1789760785.606501:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503852544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.606555:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78835712, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25990 00010000:00010000:2.0:1789760785.606563:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.606725:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.606731:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.606738:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.606752:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.612874:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986554 is committed 00002000:00010000:2.0:1789760785.613053:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78835712 -> 78839808 00002000:00010000:2.0:1789760785.613058:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154000 -> 154008 00010000:00010000:2.0:1789760785.613071:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.613083:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.613092:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.613099:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.613103:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.613112:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.613132:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.613139:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1652 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78835712->78839807] (req 78835712->78839807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.613236:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503852544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6734us (8266us total) trans 4294986554 rc 0/0 00000100:00100000:2.0:1789760785.613246:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19258 00000100:00100000:2.0:1789760785.613255:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715712431243 time 715731093219 state 0x2 ref 0 00000100:00100000:3.0:1789760785.616073:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760785.616325:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852672 00000100:00100000:0.0:1789760785.616334:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852672 00000100:00100000:0.0:1789760785.616383:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715712431243 time 715731093219 state 0x0 ref 1 00000100:00100000:0.0:1789760785.616478:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715731353635 time 715731093219 state 0x1 ref 1 00000100:00100000:0.0:1789760785.616552:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715731353635 time 715741157878 state 0x0 ref 1 00000100:00100000:0.0:1789760785.616566:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760785.616620:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78839808, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x25998 00010000:00010000:0.0:1789760785.616629:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.616646:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760785.616650:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.616656:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.616668:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.618849:0:3636:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760785.618871:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7e18a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523612416:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760785.618935:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760785.619108:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612416 00000100:00100000:3.0:1789760785.619116:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612416 00000100:00100000:3.0:1789760785.619165:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1789760785.619175:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523612416:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760785.619195:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3844878336 avail 3611951104 00000100:00100000:3.0:1789760785.619243:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523612416:12345-0@lo:13:osp-pre-0-1.0 Request processed in 69us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760785.619249:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1789760785.619308:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7e18a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523612416:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760785.619741:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760785.619751:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc8fea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523612544:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760785.619785:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760785.620055:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612544 00000100:00100000:1.0:1789760785.620061:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612544 00000100:00100000:1.0:1789760785.620141:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760785.620150:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523612544:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760785.620165:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3844878336 avail 3611951104 00000100:00100000:1.0:1789760785.620202:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3639:x1876699523612544:12345-0@lo:13:osp-pre-0-0.0 Request processed in 54us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760785.620209:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789760785.620245:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8fea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3639:1876699523612544:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760785.626121:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986555 is committed 00002000:00010000:3.0:1789760785.626232:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78839808 -> 78843904 00002000:00010000:3.0:1789760785.626238:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154008 -> 154016 00010000:00010000:3.0:1789760785.626251:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.626272:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.626281:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.626287:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760785.626292:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.626301:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.626320:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.626328:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1659 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78839808->78843903] (req 78839808->78843903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.626456:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9889us (10388us total) trans 4294986555 rc 0/0 00000100:00100000:3.0:1789760785.626473:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19259 00000100:00100000:3.0:1789760785.626797:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715731353635 time 715741157878 state 0x2 ref 0 00000100:00100000:0.0:1789760785.630723:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.630771:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852800 00000100:00100000:1.0:1789760785.630783:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852800 00000100:00100000:1.0:1789760785.630837:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715731353635 time 715741157878 state 0x0 ref 1 00000100:00100000:1.0:1789760785.630856:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715741418294 time 715741157878 state 0x1 ref 1 00000100:00100000:1.0:1789760785.630928:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715741418294 time 715755533474 state 0x0 ref 1 00000100:00100000:1.0:1789760785.630940:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.630994:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78843904, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259a0 00010000:00010000:1.0:1789760785.631004:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.631019:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.631022:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.631028:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.631040:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.635350:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986556 is committed 00002000:00010000:1.0:1789760785.636810:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78843904 -> 78848000 00002000:00010000:1.0:1789760785.636862:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154016 -> 154024 00010000:00010000:1.0:1789760785.636877:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.636890:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.636899:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.636909:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.636914:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.636923:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.636942:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.636950:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1660 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78843904->78847999] (req 78843904->78847999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.637221:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6276us (6501us total) trans 4294986556 rc 0/0 00000100:00100000:1.0:1789760785.637232:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19260 00000100:00100000:1.0:1789760785.637242:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715741418294 time 715755533474 state 0x2 ref 0 00000100:00100000:1.0:1789760785.646182:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.646465:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503852928 00000100:00100000:2.0:1789760785.646475:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503852928 00000100:00100000:2.0:1789760785.646530:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715741418294 time 715755533474 state 0x0 ref 1 00000100:00100000:2.0:1789760785.646555:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715755793890 time 715755533474 state 0x1 ref 1 00000100:00100000:2.0:1789760785.646590:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715755793890 time 715771197138 state 0x0 ref 1 00000100:00100000:2.0:1789760785.646601:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.646649:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78848000, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259a8 00010000:00010000:2.0:1789760785.646659:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.646674:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.646679:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.646685:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.646699:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.659710:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986557 is committed 00002000:00010000:3.0:1789760785.665282:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78848000 -> 78852096 00002000:00010000:3.0:1789760785.665292:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154024 -> 154032 00010000:00010000:3.0:1789760785.665305:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.665319:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.665330:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.665337:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760785.665342:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.665350:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.665369:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.665377:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1667 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78848000->78852095] (req 78848000->78852095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.665507:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503852928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18902us (19329us total) trans 4294986557 rc 0/0 00000100:00100000:3.0:1789760785.665518:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19261 00000100:00100000:3.0:1789760785.665527:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715755793890 time 715771197138 state 0x2 ref 0 00000100:00100000:0.0:1789760785.678120:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.678221:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853056 00000100:00100000:2.0:1789760785.678231:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853056 00000100:00100000:2.0:1789760785.678432:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715755793890 time 715771197138 state 0x0 ref 1 00000100:00100000:2.0:1789760785.678449:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715771457554 time 715771197138 state 0x1 ref 1 00000100:00100000:2.0:1789760785.678485:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715771457554 time 715803091309 state 0x0 ref 1 00000100:00100000:2.0:1789760785.678497:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.679216:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78852096, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259b0 00010000:00010000:2.0:1789760785.679228:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.679245:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.679250:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.679256:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.679277:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.687602:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986558 is committed 00002000:00010000:1.0:1789760785.687754:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78852096 -> 78856192 00002000:00010000:1.0:1789760785.687759:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154032 -> 154040 00010000:00010000:1.0:1789760785.687780:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.687796:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.687805:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.687812:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.687817:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.687828:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.687845:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.687852:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f166e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78852096->78856191] (req 78852096->78856191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.687995:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9494us (9880us total) trans 4294986558 rc 0/0 00000100:00100000:1.0:1789760785.688006:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19262 00000100:00100000:1.0:1789760785.688015:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715771457554 time 715803091309 state 0x2 ref 0 00000100:00100000:0.0:1789760785.696698:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.697349:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853184 00000100:00100000:3.0:1789760785.697644:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853184 00000100:00100000:3.0:1789760785.697701:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715771457554 time 715803091309 state 0x0 ref 1 00000100:00100000:3.0:1789760785.697717:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715803351725 time 715803091309 state 0x1 ref 1 00000100:00100000:3.0:1789760785.697785:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715803351725 time 715822390897 state 0x0 ref 1 00000100:00100000:3.0:1789760785.697800:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.698229:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78856192, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259b8 00010000:00010000:3.0:1789760785.698240:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.698255:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760785.698260:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.698266:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.698280:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.712460:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986559 is committed 00002000:00010000:2.0:1789760785.712814:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78856192 -> 78860288 00002000:00010000:2.0:1789760785.712825:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154040 -> 154048 00010000:00010000:2.0:1789760785.712838:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712851:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712864:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712870:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.712875:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712894:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712920:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712927:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1675 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78856192->78860287] (req 78856192->78860287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.713037:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15235us (16345us total) trans 4294986559 rc 0/0 00000100:00100000:2.0:1789760785.713048:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19263 00000100:00100000:2.0:1789760785.713057:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715803351725 time 715822390897 state 0x2 ref 0 00000100:00100000:3.0:1789760785.719635:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.719774:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853312 00000100:00100000:1.0:1789760785.719785:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853312 00000100:00100000:1.0:1789760785.719840:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715803351725 time 715822390897 state 0x0 ref 1 00000100:00100000:1.0:1789760785.719854:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715822651313 time 715822390897 state 0x1 ref 1 00000100:00100000:1.0:1789760785.719889:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715822651313 time 715844495576 state 0x0 ref 1 00000100:00100000:1.0:1789760785.719901:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503853312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.719947:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78860288, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259c0 00010000:00010000:1.0:1789760785.719956:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.719970:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.719974:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.719980:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.719993:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.732237:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986560 is committed 00002000:00010000:1.0:1789760785.732591:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78860288 -> 78864384 00002000:00010000:1.0:1789760785.732597:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154048 -> 154056 00010000:00010000:1.0:1789760785.732610:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.732621:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.732634:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.732641:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.732646:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.732654:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.732673:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.732680:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f167c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78860288->78864383] (req 78860288->78864383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.732771:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503853312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12867us (13142us total) trans 4294986560 rc 0/0 00000100:00100000:1.0:1789760785.732781:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19264 00000100:00100000:1.0:1789760785.732791:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715822651313 time 715844495576 state 0x2 ref 0 00000100:00100000:3.0:1789760785.740757:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.741169:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853440 00000100:00100000:1.0:1789760785.741178:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853440 00000100:00100000:1.0:1789760785.741300:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715822651313 time 715844495576 state 0x0 ref 1 00000100:00100000:1.0:1789760785.741315:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715844755992 time 715844495576 state 0x1 ref 1 00000100:00100000:1.0:1789760785.741420:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715844755992 time 715865995983 state 0x0 ref 1 00000100:00100000:1.0:1789760785.741434:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.741481:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78864384, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259c8 00010000:00010000:1.0:1789760785.741490:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.741505:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.741509:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.741515:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.741528:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.748946:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986561 is committed 00002000:00010000:1.0:1789760785.749146:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78864384 -> 78868480 00002000:00010000:1.0:1789760785.749151:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154056 -> 154064 00010000:00010000:1.0:1789760785.749166:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.749179:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.749187:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.749330:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.749337:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.749347:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.749371:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.749378:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1683 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78864384->78868479] (req 78864384->78868479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.750913:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9472us (10155us total) trans 4294986561 rc 0/0 00000100:00100000:1.0:1789760785.750928:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19265 00000100:00100000:1.0:1789760785.750937:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715844755992 time 715865995983 state 0x2 ref 0 00000100:00100000:0.0:1789760785.758911:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.759436:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853568 00000100:00100000:2.0:1789760785.759448:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853568 00000100:00100000:2.0:1789760785.759504:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715844755992 time 715865995983 state 0x0 ref 1 00000100:00100000:2.0:1789760785.759523:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715866256399 time 715865995983 state 0x1 ref 1 00000100:00100000:2.0:1789760785.759562:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715866256399 time 715884168085 state 0x0 ref 1 00000100:00100000:2.0:1789760785.759574:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503853568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.759629:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78868480, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259d0 00010000:00010000:2.0:1789760785.759638:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.759654:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.759658:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.759665:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.759679:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.766378:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986562 is committed 00002000:00010000:2.0:1789760785.766567:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78868480 -> 78872576 00002000:00010000:2.0:1789760785.766572:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154064 -> 154072 00010000:00010000:2.0:1789760785.766594:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.766607:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.766615:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.766624:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.766628:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.766637:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.766656:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.766663:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f168a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78868480->78872575] (req 78868480->78872575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.766760:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503853568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7182us (7852us total) trans 4294986562 rc 0/0 00000100:00100000:2.0:1789760785.766769:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19266 00000100:00100000:2.0:1789760785.766778:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715866256399 time 715884168085 state 0x2 ref 0 00000100:00100000:3.0:1789760785.771015:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.771097:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853696 00000100:00100000:2.0:1789760785.771106:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853696 00000100:00100000:2.0:1789760785.771159:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715866256399 time 715884168085 state 0x0 ref 1 00000100:00100000:2.0:1789760785.771173:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715884428501 time 715884168085 state 0x1 ref 1 00000100:00100000:2.0:1789760785.771212:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715884428501 time 715895817922 state 0x0 ref 1 00000100:00100000:2.0:1789760785.771227:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503853696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.771284:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78872576, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259d8 00010000:00010000:2.0:1789760785.771293:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.771310:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.771314:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.771320:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.771332:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.780081:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986563 is committed 00002000:00010000:2.0:1789760785.780776:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78872576 -> 78876672 00002000:00010000:2.0:1789760785.780783:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154072 -> 154080 00010000:00010000:2.0:1789760785.780797:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.780961:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.780971:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.780978:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.780983:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.780992:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.781011:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.781018:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1691 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78872576->78876671] (req 78872576->78876671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.781111:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503853696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9882us (10102us total) trans 4294986563 rc 0/0 00000100:00100000:2.0:1789760785.781122:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19267 00000100:00100000:2.0:1789760785.781133:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715884428501 time 715895817922 state 0x2 ref 0 00000100:00100000:0.0:1789760785.788964:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.789147:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853824 00000100:00100000:1.0:1789760785.789157:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853824 00000100:00100000:1.0:1789760785.789219:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715884428501 time 715895817922 state 0x0 ref 1 00000100:00100000:1.0:1789760785.789234:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715896078338 time 715895817922 state 0x1 ref 1 00000100:00100000:1.0:1789760785.789272:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715896078338 time 715913878408 state 0x0 ref 1 00000100:00100000:1.0:1789760785.789287:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.789336:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78876672, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x259e0 00010000:00010000:1.0:1789760785.789345:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.789361:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.789366:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.789372:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.789385:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.797478:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986564 is committed 00002000:00010000:1.0:1789760785.797708:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78876672 -> 78880768 00002000:00010000:1.0:1789760785.797713:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:1.0:1789760785.797715:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00002000:00010000:1.0:1789760785.797718:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154080 -> 154088 00010000:00010000:1.0:1789760785.797732:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797745:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797754:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797768:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.797772:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797781:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797800:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.798728:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1698 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78876672->78880767] (req 78876672->78880767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.798960:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503853824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9668us (9997us total) trans 4294986564 rc 0/0 00000100:00100000:1.0:1789760785.798971:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19268 00000100:00100000:1.0:1789760785.798981:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715896078338 time 715913878408 state 0x2 ref 0 00000100:00100000:0.0:1789760785.813284:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.813436:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503853952 00000100:00100000:1.0:1789760785.813447:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503853952 00000100:00100000:1.0:1789760785.813500:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715896078338 time 715913878408 state 0x0 ref 1 00000100:00100000:1.0:1789760785.813514:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715914138824 time 715913878408 state 0x1 ref 1 00000100:00100000:1.0:1789760785.813560:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715914138824 time 715938166463 state 0x0 ref 1 00000100:00100000:1.0:1789760785.813586:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503853952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.813639:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78880768, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x259e8 00010000:00010000:1.0:1789760785.813648:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.813662:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.813666:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.813673:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.813686:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.819319:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986565 is committed 00002000:00010000:1.0:1789760785.819459:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78880768 -> 78884864 00002000:00010000:1.0:1789760785.819465:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154088 -> 154096 00010000:00010000:1.0:1789760785.819477:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.819490:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.819500:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.819507:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.819513:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.819522:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.819553:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.819562:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f169f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78880768->78884863] (req 78880768->78884863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.819661:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503853952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6083us (6383us total) trans 4294986565 rc 0/0 00000100:00100000:1.0:1789760785.819671:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19269 00000100:00100000:1.0:1789760785.819681:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715914138824 time 715938166463 state 0x2 ref 0 00000100:00100000:2.0:1789760785.823869:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.824046:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854080 00000100:00100000:1.0:1789760785.824055:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854080 00000100:00100000:1.0:1789760785.824108:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715914138824 time 715938166463 state 0x0 ref 1 00000100:00100000:1.0:1789760785.824123:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715938426879 time 715938166463 state 0x1 ref 1 00000100:00100000:1.0:1789760785.824154:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715938426879 time 715948760308 state 0x0 ref 1 00000100:00100000:1.0:1789760785.824166:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503854080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.824213:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78884864, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x259f0 00010000:00010000:1.0:1789760785.824222:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824237:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760785.824242:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.824249:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824733:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.833716:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986566 is committed 00002000:00010000:1.0:1789760785.834020:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78884864 -> 78888960 00002000:00010000:1.0:1789760785.834026:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154096 -> 154104 00010000:00010000:1.0:1789760785.834040:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.834052:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.834061:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.834068:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.834073:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.834083:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.834103:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.834109:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f16a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78884864->78888959] (req 78884864->78888959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.834200:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503854080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10031us (10335us total) trans 4294986566 rc 0/0 00000100:00100000:1.0:1789760785.834210:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19270 00000100:00100000:1.0:1789760785.834219:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715938426879 time 715948760308 state 0x2 ref 0 00000100:00100000:0.0:1789760785.843206:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.843298:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854208 00000100:00100000:2.0:1789760785.843307:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854208 00000100:00100000:2.0:1789760785.843360:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715938426879 time 715948760308 state 0x0 ref 1 00000100:00100000:2.0:1789760785.843375:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715949020724 time 715948760308 state 0x1 ref 1 00000100:00100000:2.0:1789760785.843440:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715949020724 time 715968046500 state 0x0 ref 1 00000100:00100000:2.0:1789760785.843453:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503854208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.843498:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78888960, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x259f8 00010000:00010000:2.0:1789760785.843507:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.843522:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760785.843527:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.843533:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.843648:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.851247:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986567 is committed 00002000:00010000:2.0:1789760785.852700:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78888960 -> 78893056 00002000:00010000:2.0:1789760785.852707:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154104 -> 154112 00010000:00010000:2.0:1789760785.852720:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.852731:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.852739:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.852746:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.852751:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.852760:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.852792:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.852800:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f16ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78888960->78893055] (req 78888960->78893055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.852901:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503854208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9445us (9699us total) trans 4294986567 rc 0/0 00000100:00100000:2.0:1789760785.852912:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19271 00000100:00100000:2.0:1789760785.852921:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715949020724 time 715968046500 state 0x2 ref 0 00000100:00100000:0.0:1789760785.872255:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.872346:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854336 00000100:00100000:3.0:1789760785.872356:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854336 00000100:00100000:3.0:1789760785.872448:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715949020724 time 715968046500 state 0x0 ref 1 00000100:00100000:3.0:1789760785.872463:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715968306916 time 715968046500 state 0x1 ref 1 00000100:00100000:3.0:1789760785.872493:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715968306916 time 715997099919 state 0x0 ref 1 00000100:00100000:3.0:1789760785.872505:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503854336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.872569:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78893056, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a00 00010000:00010000:3.0:1789760785.872578:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.872594:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760785.872598:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.872605:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.872620:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.878450:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986568 is committed 00002000:00010000:2.0:1789760785.878620:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78893056 -> 78897152 00002000:00010000:2.0:1789760785.878626:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154112 -> 154120 00010000:00010000:2.0:1789760785.878638:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878650:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878658:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878670:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.878675:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878684:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878705:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878712:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f16b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78893056->78897151] (req 78893056->78897151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.878928:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503854336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6420us (6678us total) trans 4294986568 rc 0/0 00000100:00100000:2.0:1789760785.878939:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19272 00000100:00100000:2.0:1789760785.878948:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715968306916 time 715997099919 state 0x2 ref 0 00000100:00100000:1.0:1789760785.881792:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.881877:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854464 00000100:00100000:3.0:1789760785.881888:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854464 00000100:00100000:3.0:1789760785.881938:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715968306916 time 715997099919 state 0x0 ref 1 00000100:00100000:3.0:1789760785.881952:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715997360335 time 715997099919 state 0x1 ref 1 00000100:00100000:3.0:1789760785.881983:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715997360335 time 716006590033 state 0x0 ref 1 00000100:00100000:3.0:1789760785.881996:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503854464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.882042:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78897152, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a08 00010000:00010000:3.0:1789760785.882052:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.882066:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760785.882072:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.882078:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.882092:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.886842:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986569 is committed 00002000:00010000:3.0:1789760785.886955:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78897152 -> 78901248 00002000:00010000:3.0:1789760785.886961:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154120 -> 154128 00010000:00010000:3.0:1789760785.886973:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.886986:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.886995:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.887002:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760785.887007:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.887022:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.887042:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.887051:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78897152->78901247] (req 78897152->78901247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.887140:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503854464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5141us (5352us total) trans 4294986569 rc 0/0 00000100:00100000:3.0:1789760785.887150:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19273 00000100:00100000:3.0:1789760785.887159:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715997360335 time 716006590033 state 0x2 ref 0 00000100:00100000:0.0:1789760785.893706:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.893799:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854592 00000100:00100000:1.0:1789760785.893810:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854592 00000100:00100000:1.0:1789760785.893862:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 715997360335 time 716006590033 state 0x0 ref 1 00000100:00100000:1.0:1789760785.893877:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716006850449 time 716006590033 state 0x1 ref 1 00000100:00100000:1.0:1789760785.893908:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716006850449 time 716018514958 state 0x0 ref 1 00000100:00100000:1.0:1789760785.893920:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503854592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.893967:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78901248, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a10 00010000:00010000:1.0:1789760785.893977:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.893998:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760785.894002:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.894009:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.894022:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.898050:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986570 is committed 00002000:00010000:1.0:1789760785.898972:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78901248 -> 78905344 00002000:00010000:1.0:1789760785.898978:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154128 -> 154136 00010000:00010000:1.0:1789760785.898992:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.899004:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.899012:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.899020:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.899025:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.899033:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.899057:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.899064:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78901248->78905343] (req 78901248->78905343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.899215:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503854592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5292us (5515us total) trans 4294986570 rc 0/0 00000100:00100000:1.0:1789760785.899226:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19274 00000100:00100000:1.0:1789760785.899235:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716006850449 time 716018514958 state 0x2 ref 0 00000100:00100000:0.0:1789760785.905840:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.905940:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854720 00000100:00100000:2.0:1789760785.905949:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854720 00000100:00100000:2.0:1789760785.906006:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716006850449 time 716018514958 state 0x0 ref 1 00000100:00100000:2.0:1789760785.906020:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716018775374 time 716018514958 state 0x1 ref 1 00000100:00100000:2.0:1789760785.906084:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716018775374 time 716030690812 state 0x0 ref 1 00000100:00100000:2.0:1789760785.906100:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc17c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503854720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.906144:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78905344, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a18 00010000:00010000:2.0:1789760785.906153:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.906168:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760785.906173:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.906179:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.906193:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.910799:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986571 is committed 00002000:00010000:2.0:1789760785.910913:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78905344 -> 78909440 00002000:00010000:2.0:1789760785.910918:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154136 -> 154144 00010000:00010000:2.0:1789760785.910931:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910943:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910951:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910958:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.910964:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910973:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910994:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.911001:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78905344->78909439] (req 78905344->78909439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.911119:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc17c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503854720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5017us (5282us total) trans 4294986571 rc 0/0 00000100:00100000:2.0:1789760785.911129:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19275 00000100:00100000:2.0:1789760785.911138:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716018775374 time 716030690812 state 0x2 ref 0 00000100:00100000:2.0:1789760785.914802:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.914888:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854848 00000100:00100000:1.0:1789760785.914897:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854848 00000100:00100000:1.0:1789760785.914952:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716018775374 time 716030690812 state 0x0 ref 1 00000100:00100000:1.0:1789760785.914967:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716030951228 time 716030690812 state 0x1 ref 1 00000100:00100000:1.0:1789760785.914998:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716030951228 time 716039605298 state 0x0 ref 1 00000100:00100000:1.0:1789760785.915010:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503854848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.915054:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78909440, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a20 00010000:00010000:1.0:1789760785.915062:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.915077:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760785.915082:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.915088:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.915100:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.918730:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986572 is committed 00002000:00010000:1.0:1789760785.918876:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78909440 -> 78913536 00002000:00010000:1.0:1789760785.918881:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154144 -> 154152 00010000:00010000:1.0:1789760785.918894:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.918906:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.918915:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.918923:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.918928:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.918938:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.918958:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.918971:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f16d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78909440->78913535] (req 78909440->78913535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.919105:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503854848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4091us (4306us total) trans 4294986572 rc 0/0 00000100:00100000:1.0:1789760785.919118:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19276 00000100:00100000:1.0:1789760785.919129:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716030951228 time 716039605298 state 0x2 ref 0 00000100:00100000:0.0:1789760785.922924:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.923084:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503854976 00000100:00100000:3.0:1789760785.923093:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503854976 00000100:00100000:3.0:1789760785.923144:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716030951228 time 716039605298 state 0x0 ref 1 00000100:00100000:3.0:1789760785.923158:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716039865714 time 716039605298 state 0x1 ref 1 00000100:00100000:3.0:1789760785.923231:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716039865714 time 716047837414 state 0x0 ref 1 00000100:00100000:3.0:1789760785.923243:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503854976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.923287:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78913536, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a28 00010000:00010000:3.0:1789760785.923297:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.923312:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760785.923316:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.923322:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.923335:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.927422:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986573 is committed 00002000:00010000:3.0:1789760785.927512:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78913536 -> 78917632 00002000:00010000:3.0:1789760785.927517:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154152 -> 154160 00010000:00010000:3.0:1789760785.927530:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927559:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927568:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927576:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760785.927581:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927590:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927615:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927623:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78913536->78917631] (req 78913536->78917631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.927709:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503854976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4463us (4789us total) trans 4294986573 rc 0/0 00000100:00100000:3.0:1789760785.927718:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19277 00000100:00100000:3.0:1789760785.927728:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716039865714 time 716047837414 state 0x2 ref 0 00000100:00100000:1.0:1789760785.931629:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.931793:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503855104 00000100:00100000:2.0:1789760785.931804:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503855104 00000100:00100000:2.0:1789760785.931861:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716039865714 time 716047837414 state 0x0 ref 1 00000100:00100000:2.0:1789760785.931877:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716048097830 time 716047837414 state 0x1 ref 1 00000100:00100000:2.0:1789760785.931909:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716048097830 time 716056515557 state 0x0 ref 1 00000100:00100000:2.0:1789760785.931922:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503855104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.931971:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78917632, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a30 00010000:00010000:2.0:1789760785.931981:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.931996:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760785.932000:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.932007:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.932020:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.935420:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986574 is committed 00002000:00010000:2.0:1789760785.935530:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78917632 -> 78921728 00002000:00010000:2.0:1789760785.935544:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154160 -> 154168 00010000:00010000:2.0:1789760785.935566:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.935587:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.935596:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.935603:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.935608:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.935622:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.935639:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.935648:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f16de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78917632->78921727] (req 78917632->78921727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.935734:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503855104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3810us (4109us total) trans 4294986574 rc 0/0 00000100:00100000:2.0:1789760785.935742:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19278 00000100:00100000:2.0:1789760785.935751:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716048097830 time 716056515557 state 0x2 ref 0 00010000:00010000:3.0:1789760785.938684:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760785.939996:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.940077:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503855232 00000100:00100000:2.0:1789760785.940088:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503855232 00000100:00100000:2.0:1789760785.940139:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716048097830 time 716056515557 state 0x0 ref 1 00000100:00100000:2.0:1789760785.940152:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716056775973 time 716056515557 state 0x1 ref 1 00000100:00100000:2.0:1789760785.940181:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716056775973 time 716064787983 state 0x0 ref 1 00000100:00100000:2.0:1789760785.940192:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.940246:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78921728, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a38 00010000:00010000:2.0:1789760785.940255:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.940278:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760785.940283:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.940289:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.940303:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.943484:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986575 is committed 00002000:00010000:2.0:1789760785.943711:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78921728 -> 78925824 00002000:00010000:2.0:1789760785.943717:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154168 -> 154176 00010000:00010000:2.0:1789760785.943728:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.943740:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.943750:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.943757:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.943761:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.943770:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.943792:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.943799:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f16e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78921728->78925823] (req 78921728->78925823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.943878:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3682us (3884us total) trans 4294986575 rc 0/0 00000100:00100000:2.0:1789760785.943887:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19279 00000100:00100000:2.0:1789760785.943896:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716056775973 time 716064787983 state 0x2 ref 0 00000100:00100000:0.0:1789760785.947728:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.947926:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503855360 00000100:00100000:1.0:1789760785.947936:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503855360 00000100:00100000:1.0:1789760785.947989:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716056775973 time 716064787983 state 0x0 ref 1 00000100:00100000:1.0:1789760785.948004:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716065048399 time 716064787983 state 0x1 ref 1 00000100:00100000:1.0:1789760785.948034:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716065048399 time 716072641024 state 0x0 ref 1 00000100:00100000:1.0:1789760785.948045:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503855360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.948091:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78925824, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a40 00010000:00010000:1.0:1789760785.948099:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.948113:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760785.948119:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.948125:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.948138:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.952135:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986576 is committed 00002000:00010000:0.0:1789760785.959701:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78925824 -> 78929920 00002000:00010000:0.0:1789760785.959711:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154176 -> 154184 00010000:00010000:0.0:1789760785.959725:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.959736:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.959751:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.959758:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760785.959763:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.959772:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.959791:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.959799:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f16ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78925824->78929919] (req 78925824->78929919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.959940:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503855360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11891us (12216us total) trans 4294986576 rc 0/0 00000100:00100000:0.0:1789760785.959950:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19280 00000100:00100000:0.0:1789760785.959960:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716065048399 time 716072641024 state 0x2 ref 0 00000100:00100000:1.0:1789760785.968015:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760785.968097:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503855488 00000100:00100000:3.0:1789760785.968107:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503855488 00000100:00100000:3.0:1789760785.968160:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716065048399 time 716072641024 state 0x0 ref 1 00000100:00100000:3.0:1789760785.968175:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716072901440 time 716072641024 state 0x1 ref 1 00000100:00100000:3.0:1789760785.968209:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716072901440 time 716092815613 state 0x0 ref 1 00000100:00100000:3.0:1789760785.968221:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760785.968374:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78929920, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a48 00010000:00010000:3.0:1789760785.968383:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.968491:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760785.968495:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.968501:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.968516:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.972845:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986577 is committed 00002000:00010000:3.0:1789760785.973045:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78929920 -> 78934016 00002000:00010000:3.0:1789760785.973050:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154184 -> 154192 00010000:00010000:3.0:1789760785.973063:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.973074:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.973083:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.973090:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760785.973095:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.973105:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.973130:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.973138:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f16f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78929920->78934015] (req 78929920->78934015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.973276:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5051us (5266us total) trans 4294986577 rc 0/0 00000100:00100000:3.0:1789760785.973287:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19281 00000100:00100000:3.0:1789760785.973297:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716072901440 time 716092815613 state 0x2 ref 0 00000100:00100000:2.0:1789760785.978626:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760785.978714:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503855616 00000100:00100000:1.0:1789760785.978724:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503855616 00000100:00100000:1.0:1789760785.978778:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716072901440 time 716092815613 state 0x0 ref 1 00000100:00100000:1.0:1789760785.978794:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716093076029 time 716092815613 state 0x1 ref 1 00000100:00100000:1.0:1789760785.978826:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716093076029 time 716103431672 state 0x0 ref 1 00000100:00100000:1.0:1789760785.978838:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760785.978887:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78934016, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a50 00010000:00010000:1.0:1789760785.978895:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.978916:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760785.978920:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.978926:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.978940:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.984660:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986578 is committed 00002000:00010000:1.0:1789760785.984780:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78934016 -> 78938112 00002000:00010000:1.0:1789760785.984786:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154192 -> 154200 00010000:00010000:1.0:1789760785.984801:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.984814:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.984823:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.984830:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760785.984834:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.984843:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.984864:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.984871:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f16fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78934016->78938111] (req 78934016->78938111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.985000:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6159us (6378us total) trans 4294986578 rc 0/0 00000100:00100000:1.0:1789760785.985011:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19282 00000100:00100000:1.0:1789760785.985021:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716093076029 time 716103431672 state 0x2 ref 0 00000100:00100000:0.0:1789760785.991478:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760785.991670:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503855744 00000100:00100000:2.0:1789760785.991679:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503855744 00000100:00100000:2.0:1789760785.991736:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716093076029 time 716103431672 state 0x0 ref 1 00000100:00100000:2.0:1789760785.991751:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716103692088 time 716103431672 state 0x1 ref 1 00000100:00100000:2.0:1789760785.991783:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716103692088 time 716116389752 state 0x0 ref 1 00000100:00100000:2.0:1789760785.991801:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503855744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760785.991847:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78938112, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a58 00010000:00010000:2.0:1789760785.991856:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.991877:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760785.991882:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.991888:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.991902:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.998075:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986579 is committed 00002000:00010000:2.0:1789760785.998197:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78938112 -> 78942208 00002000:00010000:2.0:1789760785.998203:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154200 -> 154208 00010000:00010000:2.0:1789760785.998215:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.998226:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.998235:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.998242:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760785.998246:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.998265:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.998290:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.998298:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1701 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78938112->78942207] (req 78938112->78942207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.998523:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503855744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6585us (6916us total) trans 4294986579 rc 0/0 00000100:00100000:2.0:1789760785.998543:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19283 00000100:00100000:2.0:1789760785.998671:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716103692088 time 716116389752 state 0x2 ref 0 00000100:00100000:0.0:1789760786.002022:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.002808:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503855872 00000100:00100000:1.0:1789760786.002895:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503855872 00000100:00080000:3.0:1789760786.002948:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760786.002968:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716103692088 time 716116389752 state 0x0 ref 1 00000100:00100000:1.0:1789760786.003052:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716116650168 time 716116389752 state 0x1 ref 1 00000100:00100000:1.0:1789760786.003082:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716116650168 time 716127688036 state 0x0 ref 1 00000100:00100000:1.0:1789760786.003097:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.003144:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78942208, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a60 00010000:00010000:1.0:1789760786.003153:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.003169:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.003174:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.003180:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.003194:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.010838:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986580 is committed 00002000:00010000:1.0:1789760786.011304:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78942208 -> 78946304 00002000:00010000:1.0:1789760786.011309:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154208 -> 154216 00010000:00010000:1.0:1789760786.011322:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.011333:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.011341:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.011348:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.011353:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.011362:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.011381:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.011483:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1708 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78942208->78946303] (req 78942208->78946303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.011793:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503855872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8695us (9774us total) trans 4294986580 rc 0/0 00000100:00100000:0.0:1789760786.011807:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19284 00000100:00100000:0.0:1789760786.011826:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716116650168 time 716127688036 state 0x2 ref 0 00000100:00100000:0.0:1789760786.014809:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.014898:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856000 00000100:00100000:3.0:1789760786.014908:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856000 00000100:00100000:3.0:1789760786.014959:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716116650168 time 716127688036 state 0x0 ref 1 00000100:00100000:3.0:1789760786.014983:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716127948452 time 716127688036 state 0x1 ref 1 00000100:00100000:3.0:1789760786.015014:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716127948452 time 716139621251 state 0x0 ref 1 00000100:00100000:3.0:1789760786.015027:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.015072:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78946304, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a68 00010000:00010000:3.0:1789760786.015081:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.015096:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.015101:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.015107:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.015120:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.019570:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986581 is committed 00002000:00010000:3.0:1789760786.019786:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78946304 -> 78950400 00002000:00010000:3.0:1789760786.019791:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154216 -> 154224 00010000:00010000:3.0:1789760786.019804:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.019881:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.019891:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.019898:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.019904:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.019912:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.019940:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.019948:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f170f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78946304->78950399] (req 78946304->78950399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.020102:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5073us (5298us total) trans 4294986581 rc 0/0 00000100:00100000:3.0:1789760786.020113:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19285 00000100:00100000:3.0:1789760786.020123:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716127948452 time 716139621251 state 0x2 ref 0 00000100:00100000:0.0:1789760786.028062:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.028209:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856128 00000100:00100000:2.0:1789760786.028219:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856128 00000100:00100000:2.0:1789760786.028332:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716127948452 time 716139621251 state 0x0 ref 1 00000100:00100000:2.0:1789760786.028348:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716139881667 time 716139621251 state 0x1 ref 1 00000100:00100000:2.0:1789760786.028434:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716139881667 time 716153040765 state 0x0 ref 1 00000100:00100000:2.0:1789760786.028448:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.028494:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78950400, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a70 00010000:00010000:2.0:1789760786.028503:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.028519:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.028524:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.028531:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.028557:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.034158:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986582 is committed 00002000:00010000:2.0:1789760786.034718:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78950400 -> 78954496 00002000:00010000:2.0:1789760786.034724:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154224 -> 154232 00010000:00010000:2.0:1789760786.034737:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.034750:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.034758:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.034765:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.034769:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.034787:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.034807:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.035144:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1716 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78950400->78954495] (req 78950400->78954495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.035250:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6800us (7193us total) trans 4294986582 rc 0/0 00000100:00100000:2.0:1789760786.035268:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19286 00000100:00100000:2.0:1789760786.035278:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716139881667 time 716153040765 state 0x2 ref 0 00000100:00100000:0.0:1789760786.038319:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.038511:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856256 00000100:00100000:2.0:1789760786.038521:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856256 00000100:00100000:2.0:1789760786.038593:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716139881667 time 716153040765 state 0x0 ref 1 00000100:00100000:2.0:1789760786.038609:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716153301181 time 716153040765 state 0x1 ref 1 00000100:00100000:2.0:1789760786.038681:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716153301181 time 716163286899 state 0x0 ref 1 00000100:00100000:2.0:1789760786.038696:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.038743:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78954496, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a78 00010000:00010000:2.0:1789760786.038753:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038768:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.038773:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.038779:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.038797:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.044110:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986583 is committed 00002000:00010000:2.0:1789760786.044369:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78954496 -> 78958592 00002000:00010000:2.0:1789760786.044374:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154232 -> 154240 00010000:00010000:2.0:1789760786.044408:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.044420:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.045042:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.045053:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.045057:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.045067:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.045089:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.045097:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f171d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78954496->78958591] (req 78954496->78958591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.046107:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7406us (7791us total) trans 4294986583 rc 0/0 00000100:00100000:2.0:1789760786.046120:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19287 00000100:00100000:2.0:1789760786.046129:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716153301181 time 716163286899 state 0x2 ref 0 00000100:00100000:0.0:1789760786.049793:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.049869:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856384 00000100:00100000:2.0:1789760786.049878:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856384 00000100:00100000:2.0:1789760786.049940:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716153301181 time 716163286899 state 0x0 ref 1 00000100:00100000:2.0:1789760786.049956:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716163547315 time 716163286899 state 0x1 ref 1 00000100:00100000:2.0:1789760786.049987:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716163547315 time 716174593307 state 0x0 ref 1 00000100:00100000:2.0:1789760786.050000:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.050046:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78958592, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a80 00010000:00010000:2.0:1789760786.050055:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.050240:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.050246:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.050253:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.050268:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.060098:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986584 is committed 00002000:00010000:2.0:1789760786.060218:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78958592 -> 78962688 00002000:00010000:2.0:1789760786.060223:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154240 -> 154248 00010000:00010000:2.0:1789760786.060236:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.060249:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.060258:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.060265:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.060270:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.060278:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.060297:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.060305:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1724 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78958592->78962687] (req 78958592->78962687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.063206:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13193us (13407us total) trans 4294986584 rc 0/0 00000100:00100000:2.0:1789760786.063221:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19288 00000100:00100000:2.0:1789760786.063231:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716163547315 time 716174593307 state 0x2 ref 0 00000100:00100000:3.0:1789760786.067464:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.067686:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856512 00000100:00100000:1.0:1789760786.067696:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856512 00000100:00100000:1.0:1789760786.067752:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716163547315 time 716174593307 state 0x0 ref 1 00000100:00100000:1.0:1789760786.067768:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716174853723 time 716174593307 state 0x1 ref 1 00000100:00100000:1.0:1789760786.067801:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716174853723 time 716192407579 state 0x0 ref 1 00000100:00100000:1.0:1789760786.067813:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.067865:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78962688, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a88 00010000:00010000:1.0:1789760786.067874:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067890:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.067895:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.067900:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067915:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.076791:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986585 is committed 00002000:00010000:1.0:1789760786.076973:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78962688 -> 78966784 00002000:00010000:1.0:1789760786.076978:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154248 -> 154256 00010000:00010000:1.0:1789760786.076990:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.077002:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.077010:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.077017:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.077022:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.077030:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.077047:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.077055:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f172b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78962688->78966783] (req 78962688->78966783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.077183:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9368us (9725us total) trans 4294986585 rc 0/0 00000100:00100000:1.0:1789760786.077209:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19289 00000100:00100000:1.0:1789760786.077219:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716174853723 time 716192407579 state 0x2 ref 0 00000100:00100000:1.0:1789760786.082305:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.082449:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856640 00000100:00100000:2.0:1789760786.082460:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856640 00000100:00100000:2.0:1789760786.082523:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716174853723 time 716192407579 state 0x0 ref 1 00000100:00100000:2.0:1789760786.082558:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716192667995 time 716192407579 state 0x1 ref 1 00000100:00100000:2.0:1789760786.082582:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716192667995 time 716207188597 state 0x0 ref 1 00000100:00100000:2.0:1789760786.082595:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.082645:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78966784, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a90 00010000:00010000:2.0:1789760786.082655:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.082673:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.082678:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.082685:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.082701:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.090427:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986586 is committed 00002000:00010000:2.0:1789760786.090595:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78966784 -> 78970880 00002000:00010000:2.0:1789760786.090600:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154256 -> 154264 00010000:00010000:2.0:1789760786.090613:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.090626:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.090635:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.090642:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.090647:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.090657:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.090675:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.090682:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1732 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78966784->78970879] (req 78966784->78970879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.090774:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8177us (8475us total) trans 4294986586 rc 0/0 00000100:00100000:2.0:1789760786.090785:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19290 00000100:00100000:2.0:1789760786.090794:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716192667995 time 716207188597 state 0x2 ref 0 00000100:00100000:0.0:1789760786.098802:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.098912:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856768 00000100:00100000:2.0:1789760786.098921:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856768 00000100:00100000:2.0:1789760786.098977:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716192667995 time 716207188597 state 0x0 ref 1 00000100:00100000:2.0:1789760786.098992:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716207449013 time 716207188597 state 0x1 ref 1 00000100:00100000:2.0:1789760786.099065:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716207449013 time 716223671259 state 0x0 ref 1 00000100:00100000:2.0:1789760786.099077:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.099123:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78970880, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25a98 00010000:00010000:2.0:1789760786.099132:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.099148:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.099152:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.099158:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.099172:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.109145:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986587 is committed 00002000:00010000:2.0:1789760786.109608:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78970880 -> 78974976 00002000:00010000:2.0:1789760786.109613:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154264 -> 154272 00010000:00010000:2.0:1789760786.109629:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.109647:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.109656:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.109664:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.109668:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.109678:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.109698:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.109705:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1739 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78970880->78974975] (req 78970880->78974975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.110559:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503856768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11471us (11753us total) trans 4294986587 rc 0/0 00000100:00100000:2.0:1789760786.110572:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19291 00000100:00100000:2.0:1789760786.110582:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716207449013 time 716223671259 state 0x2 ref 0 00000100:00100000:3.0:1789760786.116037:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760786.116658:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503856896 00000100:00100000:0.0:1789760786.116669:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503856896 00000100:00100000:0.0:1789760786.116736:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716207449013 time 716223671259 state 0x0 ref 1 00000100:00100000:0.0:1789760786.116752:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716223931675 time 716223671259 state 0x1 ref 1 00000100:00100000:0.0:1789760786.116782:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716223931675 time 716241388457 state 0x0 ref 1 00000100:00100000:0.0:1789760786.116794:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760786.116839:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78974976, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25aa0 00010000:00010000:0.0:1789760786.116850:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.116866:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760786.116871:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.116877:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.116892:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.129968:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986588 is committed 00002000:00010000:0.0:1789760786.130090:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78974976 -> 78979072 00002000:00010000:0.0:1789760786.130096:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154272 -> 154280 00010000:00010000:0.0:1789760786.130109:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.130122:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.130130:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.130138:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760786.130142:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.130151:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.130170:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.130177:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1740 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78974976->78979071] (req 78974976->78979071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.130276:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503856896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13478us (14244us total) trans 4294986588 rc 0/0 00000100:00100000:0.0:1789760786.130287:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19292 00000100:00100000:0.0:1789760786.130297:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716223931675 time 716241388457 state 0x2 ref 0 00000100:00100000:0.0:1789760786.135799:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.135871:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857024 00000100:00100000:1.0:1789760786.135880:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857024 00000100:00100000:1.0:1789760786.135932:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716223931675 time 716241388457 state 0x0 ref 1 00000100:00100000:1.0:1789760786.135947:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716241648873 time 716241388457 state 0x1 ref 1 00000100:00100000:1.0:1789760786.135977:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716241648873 time 716260584259 state 0x0 ref 1 00000100:00100000:1.0:1789760786.135990:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.136037:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78979072, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25aa8 00010000:00010000:1.0:1789760786.136046:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.136063:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.136067:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.136073:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.136086:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.146501:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986589 is committed 00002000:00010000:1.0:1789760786.146790:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78979072 -> 78983168 00002000:00010000:1.0:1789760786.146796:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154280 -> 154288 00010000:00010000:1.0:1789760786.146819:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.146832:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.146841:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.146848:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.146854:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.146863:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.146883:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.146890:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1747 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78979072->78983167] (req 78979072->78983167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.146984:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10990us (11189us total) trans 4294986589 rc 0/0 00000100:00100000:1.0:1789760786.146994:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19293 00000100:00100000:1.0:1789760786.147004:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716241648873 time 716260584259 state 0x2 ref 0 00000100:00100000:2.0:1789760786.153496:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.153649:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857152 00000100:00100000:1.0:1789760786.153659:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857152 00000100:00100000:1.0:1789760786.153719:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716241648873 time 716260584259 state 0x0 ref 1 00000100:00100000:1.0:1789760786.153735:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716260844675 time 716260584259 state 0x1 ref 1 00000100:00100000:1.0:1789760786.153757:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716260844675 time 716278364927 state 0x0 ref 1 00000100:00100000:1.0:1789760786.153769:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.153833:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78983168, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ab0 00010000:00010000:1.0:1789760786.153843:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.153859:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.153864:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.153871:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.153884:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.160754:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986590 is committed 00002000:00010000:1.0:1789760786.160964:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78983168 -> 78987264 00002000:00010000:1.0:1789760786.160970:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154288 -> 154296 00010000:00010000:1.0:1789760786.160983:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.160996:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.161005:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.161012:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.161017:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.161026:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.161046:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.161054:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f174e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78983168->78987263] (req 78983168->78987263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.161177:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7403us (7685us total) trans 4294986590 rc 0/0 00000100:00100000:1.0:1789760786.161187:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19294 00000100:00100000:1.0:1789760786.161197:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716260844675 time 716278364927 state 0x2 ref 0 00000100:00100000:3.0:1789760786.171644:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.171821:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857280 00000100:00100000:2.0:1789760786.171831:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857280 00000100:00100000:2.0:1789760786.171887:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716260844675 time 716278364927 state 0x0 ref 1 00000100:00100000:2.0:1789760786.171903:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716278625343 time 716278364927 state 0x1 ref 1 00000100:00100000:2.0:1789760786.171934:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716278625343 time 716296540915 state 0x0 ref 1 00000100:00100000:2.0:1789760786.171946:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503857280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.172384:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78987264, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ab8 00010000:00010000:2.0:1789760786.172436:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.172452:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.172458:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.172465:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.172478:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.181851:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986591 is committed 00002000:00010000:2.0:1789760786.188624:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78987264 -> 78991360 00002000:00010000:2.0:1789760786.188635:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154296 -> 154304 00010000:00010000:2.0:1789760786.188649:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.188662:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.188672:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.188679:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.188685:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.188694:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.188713:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.188721:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1755 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78987264->78991359] (req 78987264->78991359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.188822:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503857280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16860us (17170us total) trans 4294986591 rc 0/0 00000100:00100000:2.0:1789760786.188832:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19295 00000100:00100000:2.0:1789760786.188842:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716278625343 time 716296540915 state 0x2 ref 0 00000100:00100000:0.0:1789760786.197802:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.197959:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857408 00000100:00100000:2.0:1789760786.197969:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857408 00000100:00100000:2.0:1789760786.198021:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716278625343 time 716296540915 state 0x0 ref 1 00000100:00100000:2.0:1789760786.198037:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716296801331 time 716296540915 state 0x1 ref 1 00000100:00100000:2.0:1789760786.198069:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716296801331 time 716322675356 state 0x0 ref 1 00000100:00100000:2.0:1789760786.198082:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503857408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.198134:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78991360, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ac0 00010000:00010000:2.0:1789760786.198143:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.198159:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.198164:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.198171:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.198186:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.211661:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986592 is committed 00002000:00010000:0.0:1789760786.211802:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78991360 -> 78995456 00002000:00010000:0.0:1789760786.211880:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154304 -> 154312 00010000:00010000:0.0:1789760786.211895:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.211908:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.211916:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.211923:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760786.211927:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.211936:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.211961:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.212027:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f175c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78991360->78995455] (req 78991360->78995455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.212461:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503857408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14376us (14665us total) trans 4294986592 rc 0/0 00000100:00100000:0.0:1789760786.212472:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19296 00000100:00100000:0.0:1789760786.212481:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716296801331 time 716322675356 state 0x2 ref 0 00000100:00100000:0.0:1789760786.223821:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.223935:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857536 00000100:00100000:1.0:1789760786.223945:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857536 00000100:00100000:1.0:1789760786.224011:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716296801331 time 716322675356 state 0x0 ref 1 00000100:00100000:1.0:1789760786.224027:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716322935772 time 716322675356 state 0x1 ref 1 00000100:00100000:1.0:1789760786.224058:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716322935772 time 716348664509 state 0x0 ref 1 00000100:00100000:1.0:1789760786.224070:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.224119:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78995456, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ac8 00010000:00010000:1.0:1789760786.224128:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.224144:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.224149:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.224155:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.224177:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.247654:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986593 is committed 00002000:00010000:1.0:1789760786.248117:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78995456 -> 78999552 00002000:00010000:1.0:1789760786.248122:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154312 -> 154320 00010000:00010000:1.0:1789760786.248242:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.248256:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.248325:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.248332:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.248337:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.248347:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.248368:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.248382:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1763 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78995456->78999551] (req 78995456->78999551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.248563:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 24489us (24760us total) trans 4294986593 rc 0/0 00000100:00100000:1.0:1789760786.248574:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19297 00000100:00100000:1.0:1789760786.248585:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716322935772 time 716348664509 state 0x2 ref 0 00000100:00100000:0.0:1789760786.262585:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.262747:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857664 00000100:00100000:1.0:1789760786.262757:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857664 00000100:00100000:1.0:1789760786.262823:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716322935772 time 716348664509 state 0x0 ref 1 00000100:00100000:1.0:1789760786.262838:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716348924925 time 716348664509 state 0x1 ref 1 00000100:00100000:1.0:1789760786.262867:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716348924925 time 716387473885 state 0x0 ref 1 00000100:00100000:1.0:1789760786.262880:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.262927:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 78999552, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ad0 00010000:00010000:1.0:1789760786.262936:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.262953:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.262957:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.262964:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.263480:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.272922:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986594 is committed 00002000:00010000:1.0:1789760786.273080:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 78999552 -> 79003648 00002000:00010000:1.0:1789760786.273085:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154320 -> 154328 00010000:00010000:1.0:1789760786.273098:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.273234:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.273245:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.273252:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.273361:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.273372:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.273407:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.273416:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f176a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [78999552->79003647] (req 78999552->79003647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.273562:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10679us (10983us total) trans 4294986594 rc 0/0 00000100:00100000:1.0:1789760786.273582:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19298 00000100:00100000:1.0:1789760786.273592:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716348924925 time 716387473885 state 0x2 ref 0 00000100:00100000:0.0:1789760786.276118:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.276794:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857792 00000100:00100000:2.0:1789760786.276804:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857792 00000100:00100000:2.0:1789760786.276922:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716348924925 time 716387473885 state 0x0 ref 1 00000100:00100000:2.0:1789760786.276937:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716387734301 time 716387473885 state 0x1 ref 1 00000100:00100000:2.0:1789760786.277002:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716387734301 time 716401608169 state 0x0 ref 1 00000100:00100000:2.0:1789760786.277016:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.277066:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79003648, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ad8 00010000:00010000:2.0:1789760786.277076:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.277092:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.277097:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.277103:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.277129:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.285245:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986595 is committed 00002000:00010000:2.0:1789760786.285651:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79003648 -> 79007744 00002000:00010000:2.0:1789760786.285658:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154328 -> 154336 00010000:00010000:2.0:1789760786.285672:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.285686:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.285695:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.285703:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.285709:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.285724:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.285743:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.285752:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1771 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79003648->79007743] (req 79003648->79007743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.285913:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503857792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8895us (9809us total) trans 4294986595 rc 0/0 00000100:00100000:2.0:1789760786.285923:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19299 00000100:00100000:2.0:1789760786.285933:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716387734301 time 716401608169 state 0x2 ref 0 00000100:00100000:0.0:1789760786.291247:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.292892:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503857920 00000100:00100000:2.0:1789760786.292907:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503857920 00000100:00100000:2.0:1789760786.292968:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716387734301 time 716401608169 state 0x0 ref 1 00000100:00100000:2.0:1789760786.292984:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716401868585 time 716401608169 state 0x1 ref 1 00000100:00100000:2.0:1789760786.293385:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716401868585 time 716417989193 state 0x0 ref 1 00000100:00100000:2.0:1789760786.293431:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503857920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.293482:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79007744, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ae0 00010000:00010000:2.0:1789760786.293492:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.293513:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.293517:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.293523:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.293656:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.300928:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986596 is committed 00002000:00010000:2.0:1789760786.301056:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79007744 -> 79011840 00002000:00010000:2.0:1789760786.301061:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154336 -> 154344 00010000:00010000:2.0:1789760786.301076:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301089:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301098:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301116:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.301121:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301131:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301166:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301174:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1778 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79007744->79011839] (req 79007744->79011839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.301278:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503857920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7846us (10035us total) trans 4294986596 rc 0/0 00000100:00100000:2.0:1789760786.301288:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19300 00000100:00100000:2.0:1789760786.301298:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716401868585 time 716417989193 state 0x2 ref 0 00000100:00100000:0.0:1789760786.310966:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.312914:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858048 00000100:00100000:1.0:1789760786.312930:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858048 00000100:00100000:1.0:1789760786.312991:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716401868585 time 716417989193 state 0x0 ref 1 00000100:00100000:1.0:1789760786.313007:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716418249609 time 716417989193 state 0x1 ref 1 00000100:00100000:1.0:1789760786.313092:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716418249609 time 716437697749 state 0x0 ref 1 00000100:00100000:1.0:1789760786.313117:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.313172:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79011840, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ae8 00010000:00010000:1.0:1789760786.313182:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.313208:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.313212:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.313219:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.313232:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.319461:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986597 is committed 00002000:00010000:1.0:1789760786.319839:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79011840 -> 79015936 00002000:00010000:1.0:1789760786.319844:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154344 -> 154352 00010000:00010000:1.0:1789760786.319858:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.319872:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.319881:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.319890:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.319894:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.319904:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.319924:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.319933:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f177f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79011840->79015935] (req 79011840->79015935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.320026:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6916us (9065us total) trans 4294986597 rc 0/0 00000100:00100000:1.0:1789760786.320038:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19301 00000100:00100000:1.0:1789760786.320049:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716418249609 time 716437697749 state 0x2 ref 0 00000100:00100000:0.0:1789760786.323847:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.323989:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858176 00000100:00100000:1.0:1789760786.324000:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858176 00000100:00100000:1.0:1789760786.324056:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716418249609 time 716437697749 state 0x0 ref 1 00000100:00100000:1.0:1789760786.324072:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716437958165 time 716437697749 state 0x1 ref 1 00000100:00100000:1.0:1789760786.324103:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716437958165 time 716448709998 state 0x0 ref 1 00000100:00100000:1.0:1789760786.324125:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.324437:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79015936, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25af0 00010000:00010000:1.0:1789760786.324446:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.324462:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.324467:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.324474:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.324488:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.333531:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986598 is committed 00002000:00010000:1.0:1789760786.334010:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79015936 -> 79020032 00002000:00010000:1.0:1789760786.334015:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154352 -> 154360 00010000:00010000:1.0:1789760786.334030:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.334042:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.334052:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.334058:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.334063:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.334072:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.334091:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.334098:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1786 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79015936->79020031] (req 79015936->79020031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.334757:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10627us (10913us total) trans 4294986598 rc 0/0 00000100:00100000:1.0:1789760786.334767:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19302 00000100:00100000:1.0:1789760786.334777:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716437958165 time 716448709998 state 0x2 ref 0 00000100:00100000:0.0:1789760786.342504:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.349702:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858304 00000100:00100000:2.0:1789760786.349717:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858304 00000100:00100000:2.0:1789760786.349784:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716437958165 time 716448709998 state 0x0 ref 1 00000100:00100000:2.0:1789760786.349800:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716448970414 time 716448709998 state 0x1 ref 1 00000100:00100000:2.0:1789760786.350563:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716448970414 time 716475164769 state 0x0 ref 1 00000100:00100000:2.0:1789760786.350590:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.350642:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79020032, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25af8 00010000:00010000:2.0:1789760786.350651:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.350666:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.350671:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.350678:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.350692:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.358158:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986599 is committed 00002000:00010000:2.0:1789760786.358329:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79020032 -> 79024128 00002000:00010000:2.0:1789760786.358335:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154360 -> 154368 00010000:00010000:2.0:1789760786.358348:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.358360:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.358368:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.358375:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.358380:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.358425:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.358450:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.358464:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f178d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79020032->79024127] (req 79020032->79024127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.358582:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7986us (16077us total) trans 4294986599 rc 0/0 00000100:00100000:2.0:1789760786.358592:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19303 00000100:00100000:2.0:1789760786.358602:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716448970414 time 716475164769 state 0x2 ref 0 00000100:00100000:0.0:1789760786.362676:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.362764:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858432 00000100:00100000:2.0:1789760786.362774:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858432 00000100:00100000:2.0:1789760786.362831:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716448970414 time 716475164769 state 0x0 ref 1 00000100:00100000:2.0:1789760786.362846:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716475425185 time 716475164769 state 0x1 ref 1 00000100:00100000:2.0:1789760786.362947:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716475425185 time 716487484739 state 0x0 ref 1 00000100:00100000:2.0:1789760786.362964:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.363015:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79024128, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b00 00010000:00010000:2.0:1789760786.363025:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.363103:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.363109:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.363116:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.363135:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.369278:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986600 is committed 00002000:00010000:3.0:1789760786.369572:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79024128 -> 79028224 00002000:00010000:3.0:1789760786.369578:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154368 -> 154376 00010000:00010000:3.0:1789760786.369592:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.369610:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.369620:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.369627:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.369632:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.369643:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.369663:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.369671:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1794 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79024128->79028223] (req 79024128->79028223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.369768:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6804us (7097us total) trans 4294986600 rc 0/0 00000100:00100000:3.0:1789760786.369780:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19304 00000100:00100000:3.0:1789760786.369789:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716475425185 time 716487484739 state 0x2 ref 0 00000100:00100000:0.0:1789760786.373702:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.373758:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858560 00000100:00100000:1.0:1789760786.373768:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858560 00000100:00100000:1.0:1789760786.373833:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716475425185 time 716487484739 state 0x0 ref 1 00000100:00100000:1.0:1789760786.373849:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716487745155 time 716487484739 state 0x1 ref 1 00000100:00100000:1.0:1789760786.373926:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716487745155 time 716498531862 state 0x0 ref 1 00000100:00100000:1.0:1789760786.373940:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.373993:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79028224, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b08 00010000:00010000:1.0:1789760786.374002:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.374018:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.374023:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.374029:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.374043:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.381448:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986601 is committed 00002000:00010000:1.0:1789760786.381570:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79028224 -> 79032320 00002000:00010000:1.0:1789760786.381576:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154376 -> 154384 00010000:00010000:1.0:1789760786.381591:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.381606:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.381616:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.381624:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.381629:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.381638:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.381657:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.381666:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f179b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79028224->79032319] (req 79028224->79032319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.381821:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7877us (8124us total) trans 4294986601 rc 0/0 00000100:00100000:1.0:1789760786.381832:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19305 00000100:00100000:1.0:1789760786.381842:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716487745155 time 716498531862 state 0x2 ref 0 00000100:00100000:0.0:1789760786.390086:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.390689:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858688 00000100:00100000:2.0:1789760786.390700:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858688 00000100:00100000:2.0:1789760786.390759:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716487745155 time 716498531862 state 0x0 ref 1 00000100:00100000:2.0:1789760786.390775:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716498792278 time 716498531862 state 0x1 ref 1 00000100:00100000:2.0:1789760786.390874:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716498792278 time 716515413275 state 0x0 ref 1 00000100:00100000:2.0:1789760786.390889:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.390939:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79032320, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b10 00010000:00010000:2.0:1789760786.390947:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.390963:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.390968:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.390975:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.390988:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.401447:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986602 is committed 00002000:00010000:2.0:1789760786.401741:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79032320 -> 79036416 00002000:00010000:2.0:1789760786.401747:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154384 -> 154392 00010000:00010000:2.0:1789760786.401761:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.401773:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.401781:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.401788:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.401793:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.401801:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.402490:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.402501:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79032320->79036415] (req 79032320->79036415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.402817:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11915us (12726us total) trans 4294986602 rc 0/0 00000100:00100000:2.0:1789760786.402828:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19306 00000100:00100000:2.0:1789760786.402838:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716498792278 time 716515413275 state 0x2 ref 0 00000100:00100000:0.0:1789760786.416761:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.416986:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858816 00000100:00100000:3.0:1789760786.416995:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858816 00000100:00100000:3.0:1789760786.417054:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716498792278 time 716515413275 state 0x0 ref 1 00000100:00100000:3.0:1789760786.417068:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716515673691 time 716515413275 state 0x1 ref 1 00000100:00100000:3.0:1789760786.417094:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716515673691 time 716541700990 state 0x0 ref 1 00000100:00100000:3.0:1789760786.417106:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.417154:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79036416, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b18 00010000:00010000:3.0:1789760786.417163:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.417181:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.417185:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.417192:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.417205:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.423756:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986603 is committed 00002000:00010000:3.0:1789760786.425244:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79036416 -> 79040512 00002000:00010000:3.0:1789760786.425253:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154392 -> 154400 00010000:00010000:3.0:1789760786.425278:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.425297:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.425306:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.425313:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.425318:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.425327:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.425346:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.425354:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79036416->79040511] (req 79036416->79040511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.425525:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503858816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8416us (8767us total) trans 4294986603 rc 0/0 00000100:00100000:3.0:1789760786.425544:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19307 00000100:00100000:3.0:1789760786.425556:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716515673691 time 716541700990 state 0x2 ref 0 00000100:00100000:0.0:1789760786.430992:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.433289:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503858944 00000100:00100000:3.0:1789760786.433304:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503858944 00000100:00100000:0.0:1789760786.435986:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716515673691 time 716541700990 state 0x0 ref 1 00000100:00100000:0.0:1789760786.436006:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716541961406 time 716541700990 state 0x1 ref 1 00000100:00100000:0.0:1789760786.436048:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716541961406 time 716560655330 state 0x0 ref 1 00000100:00100000:0.0:1789760786.436063:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760786.436110:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79040512, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b20 00010000:00010000:0.0:1789760786.436121:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.436137:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760786.436141:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.436148:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.436162:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.440529:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986604 is committed 00002000:00010000:0.0:1789760786.440701:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79040512 -> 79044608 00002000:00010000:0.0:1789760786.440706:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154400 -> 154408 00010000:00010000:0.0:1789760786.440718:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.440730:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.440744:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.440752:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760786.440756:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.440764:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.440783:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.440790:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f17b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79040512->79044607] (req 79040512->79044607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.440881:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503858944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4817us (9894us total) trans 4294986604 rc 0/0 00000100:00100000:0.0:1789760786.440892:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19308 00000100:00100000:0.0:1789760786.440901:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716541961406 time 716560655330 state 0x2 ref 0 00000100:00100000:0.0:1789760786.445235:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.445369:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859072 00000100:00100000:1.0:1789760786.445378:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859072 00000100:00100000:1.0:1789760786.445610:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716541961406 time 716560655330 state 0x0 ref 1 00000100:00100000:1.0:1789760786.445625:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716560915746 time 716560655330 state 0x1 ref 1 00000100:00100000:1.0:1789760786.447678:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716560915746 time 716572274915 state 0x0 ref 1 00000100:00100000:1.0:1789760786.447708:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.447766:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79044608, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b28 00010000:00010000:1.0:1789760786.447776:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.447791:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.447796:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.447802:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.447816:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.452890:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986605 is committed 00002000:00010000:1.0:1789760786.453004:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79044608 -> 79048704 00002000:00010000:1.0:1789760786.453009:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154408 -> 154416 00010000:00010000:1.0:1789760786.453023:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.453040:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.453049:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.453056:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.453062:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.453072:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.453092:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.453100:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f17b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79044608->79048703] (req 79044608->79048703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.453229:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5529us (8000us total) trans 4294986605 rc 0/0 00000100:00100000:1.0:1789760786.453240:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19309 00000100:00100000:1.0:1789760786.453249:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716560915746 time 716572274915 state 0x2 ref 0 00000100:00100000:1.0:1789760786.464725:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.468492:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859200 00000100:00100000:2.0:1789760786.468508:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859200 00000100:00100000:2.0:1789760786.468590:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716560915746 time 716572274915 state 0x0 ref 1 00000100:00100000:2.0:1789760786.468606:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716572535331 time 716572274915 state 0x1 ref 1 00000100:00100000:2.0:1789760786.468643:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716572535331 time 716593249369 state 0x0 ref 1 00000100:00100000:2.0:1789760786.468657:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.468704:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79048704, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b30 00010000:00010000:2.0:1789760786.468714:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.468729:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.468734:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.468740:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.468755:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.474957:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986606 is committed 00002000:00010000:2.0:1789760786.475125:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79048704 -> 79052800 00002000:00010000:2.0:1789760786.475135:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154416 -> 154424 00010000:00010000:2.0:1789760786.475148:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.475166:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.475175:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.475182:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.475187:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.475196:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.475214:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.475222:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79048704->79052799] (req 79048704->79052799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.475331:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6672us (10610us total) trans 4294986606 rc 0/0 00000100:00100000:2.0:1789760786.475340:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19310 00000100:00100000:2.0:1789760786.475350:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716572535331 time 716593249369 state 0x2 ref 0 00000100:00100000:0.0:1789760786.485562:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.485709:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859328 00000100:00100000:3.0:1789760786.485718:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859328 00000100:00100000:3.0:1789760786.485774:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716572535331 time 716593249369 state 0x0 ref 1 00000100:00100000:3.0:1789760786.485789:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716593509785 time 716593249369 state 0x1 ref 1 00000100:00100000:3.0:1789760786.485818:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716593509785 time 716610425020 state 0x0 ref 1 00000100:00100000:3.0:1789760786.485830:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.485879:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79052800, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b38 00010000:00010000:3.0:1789760786.485889:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.485905:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.485909:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.485916:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.485930:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.490556:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986607 is committed 00002000:00010000:3.0:1789760786.490738:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79052800 -> 79056896 00002000:00010000:3.0:1789760786.490746:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154424 -> 154432 00010000:00010000:3.0:1789760786.490758:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.490767:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.490773:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.490779:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.490782:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.490788:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.490801:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.490806:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79052800->79056895] (req 79052800->79056895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.490927:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5094us (5370us total) trans 4294986607 rc 0/0 00000100:00100000:3.0:1789760786.490938:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19311 00000100:00100000:3.0:1789760786.490951:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716593509785 time 716610425020 state 0x2 ref 0 00000100:00100000:0.0:1789760786.494160:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.494781:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859456 00000100:00100000:3.0:1789760786.494793:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859456 00000100:00100000:3.0:1789760786.494850:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716593509785 time 716610425020 state 0x0 ref 1 00000100:00100000:3.0:1789760786.494866:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716610685436 time 716610425020 state 0x1 ref 1 00000100:00100000:3.0:1789760786.494898:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716610685436 time 716619504254 state 0x0 ref 1 00000100:00100000:3.0:1789760786.494910:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.494957:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79056896, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b40 00010000:00010000:3.0:1789760786.494966:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.494980:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.494984:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.494990:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.495003:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.500590:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986608 is committed 00002000:00010000:3.0:1789760786.500750:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79056896 -> 79060992 00002000:00010000:3.0:1789760786.500756:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154432 -> 154440 00010000:00010000:3.0:1789760786.500773:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.500787:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.500797:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.500810:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.500816:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.500827:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.500853:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.500861:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79056896->79060991] (req 79056896->79060991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.500997:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6081us (6839us total) trans 4294986608 rc 0/0 00000100:00100000:3.0:1789760786.501009:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19312 00000100:00100000:3.0:1789760786.501022:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716610685436 time 716619504254 state 0x2 ref 0 00000100:00100000:1.0:1789760786.505859:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.505927:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859584 00000100:00100000:3.0:1789760786.505940:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859584 00000100:00100000:3.0:1789760786.506014:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716610685436 time 716619504254 state 0x0 ref 1 00000100:00100000:3.0:1789760786.506033:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716619764670 time 716619504254 state 0x1 ref 1 00000100:00100000:3.0:1789760786.506115:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716619764670 time 716630721346 state 0x0 ref 1 00000100:00100000:3.0:1789760786.506129:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503859584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.506182:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79060992, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b48 00010000:00010000:3.0:1789760786.506191:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.506207:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.506211:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.506216:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.506229:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.512170:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986609 is committed 00002000:00010000:3.0:1789760786.512444:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79060992 -> 79065088 00002000:00010000:3.0:1789760786.512450:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154440 -> 154448 00010000:00010000:3.0:1789760786.512467:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.512481:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.512491:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.512500:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.512506:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.512517:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.512553:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.512562:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79060992->79065087] (req 79060992->79065087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.512704:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503859584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6568us (6853us total) trans 4294986609 rc 0/0 00000100:00100000:3.0:1789760786.512717:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19313 00000100:00100000:3.0:1789760786.512729:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716619764670 time 716630721346 state 0x2 ref 0 00000100:00100000:0.0:1789760786.517073:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.517166:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859712 00000100:00100000:2.0:1789760786.517177:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859712 00000100:00100000:2.0:1789760786.517239:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716619764670 time 716630721346 state 0x0 ref 1 00000100:00100000:2.0:1789760786.517253:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716630981762 time 716630721346 state 0x1 ref 1 00000100:00100000:2.0:1789760786.517280:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716630981762 time 716641887314 state 0x0 ref 1 00000100:00100000:2.0:1789760786.517424:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503859712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.517496:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79065088, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b50 00010000:00010000:2.0:1789760786.517509:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.517527:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.517541:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.517549:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.517565:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.522632:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986610 is committed 00002000:00010000:2.0:1789760786.522846:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79065088 -> 79069184 00002000:00010000:2.0:1789760786.522853:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154448 -> 154456 00010000:00010000:2.0:1789760786.522871:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.522885:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.522899:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.522907:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.522913:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.522924:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.522949:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.522958:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f17da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79065088->79069183] (req 79065088->79069183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.523077:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503859712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5688us (6009us total) trans 4294986610 rc 0/0 00000100:00100000:2.0:1789760786.523088:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19314 00000100:00100000:2.0:1789760786.523099:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716630981762 time 716641887314 state 0x2 ref 0 00000100:00100000:0.0:1789760786.527678:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.527771:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859840 00000100:00100000:2.0:1789760786.527784:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859840 00000100:00100000:2.0:1789760786.527861:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716630981762 time 716641887314 state 0x0 ref 1 00000100:00100000:2.0:1789760786.527878:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716642147730 time 716641887314 state 0x1 ref 1 00000100:00100000:2.0:1789760786.527948:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716642147730 time 716652552887 state 0x0 ref 1 00000100:00100000:2.0:1789760786.527964:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503859840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.528033:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79069184, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b58 00010000:00010000:2.0:1789760786.528043:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.528059:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.528063:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.528068:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.528080:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.533248:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986611 is committed 00002000:00010000:2.0:1789760786.535802:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79069184 -> 79073280 00002000:00010000:2.0:1789760786.537469:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154456 -> 154464 00010000:00010000:2.0:1789760786.537497:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537514:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537524:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537547:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.537553:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537563:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537597:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537605:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79069184->79073279] (req 79069184->79073279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.537748:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503859840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9777us (10076us total) trans 4294986611 rc 0/0 00000100:00100000:2.0:1789760786.537761:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19315 00000100:00100000:2.0:1789760786.537774:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716642147730 time 716652552887 state 0x2 ref 0 00000100:00100000:1.0:1789760786.543710:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.544008:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503859968 00000100:00100000:3.0:1789760786.544021:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503859968 00000100:00100000:3.0:1789760786.544089:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716642147730 time 716652552887 state 0x0 ref 1 00000100:00100000:3.0:1789760786.544107:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716652813303 time 716652552887 state 0x1 ref 1 00000100:00100000:3.0:1789760786.544141:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716652813303 time 716668747930 state 0x0 ref 1 00000100:00100000:3.0:1789760786.544154:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.544218:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79073280, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b60 00010000:00010000:3.0:1789760786.544230:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.544250:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.544679:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.544689:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.544710:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.562665:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986612 is committed 00002000:00010000:2.0:1789760786.562949:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79073280 -> 79077376 00002000:00010000:2.0:1789760786.562956:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154464 -> 154472 00010000:00010000:2.0:1789760786.562974:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.562988:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.563003:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.563012:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.563017:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.563027:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.563054:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.563063:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f17e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79073280->79077375] (req 79073280->79077375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.563215:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503859968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19054us (19510us total) trans 4294986612 rc 0/0 00000100:00100000:2.0:1789760786.563227:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19316 00000100:00100000:2.0:1789760786.563239:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716652813303 time 716668747930 state 0x2 ref 0 00000100:00100000:0.0:1789760786.572158:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.572314:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860096 00000100:00100000:3.0:1789760786.572339:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860096 00000100:00100000:3.0:1789760786.572446:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716652813303 time 716668747930 state 0x0 ref 1 00000100:00100000:3.0:1789760786.572475:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716669008346 time 716668747930 state 0x1 ref 1 00000100:00100000:3.0:1789760786.572513:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716669008346 time 716697118754 state 0x0 ref 1 00000100:00100000:3.0:1789760786.572529:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503860096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.572618:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79077376, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b68 00010000:00010000:3.0:1789760786.572630:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.572650:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.572656:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.572664:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.572682:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.589279:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986613 is committed 00002000:00010000:3.0:1789760786.595974:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79077376 -> 79081472 00002000:00010000:3.0:1789760786.595996:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154472 -> 154480 00010000:00010000:3.0:1789760786.596015:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596031:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596041:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596049:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.596056:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596067:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596101:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596110:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f17ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79077376->79081471] (req 79077376->79081471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.596300:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503860096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 23767us (24146us total) trans 4294986613 rc 0/0 00000100:00100000:3.0:1789760786.596311:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19317 00000100:00100000:3.0:1789760786.596326:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716669008346 time 716697118754 state 0x2 ref 0 00000100:00100000:0.0:1789760786.603266:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.603495:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860224 00000100:00100000:2.0:1789760786.603517:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860224 00000100:00100000:2.0:1789760786.603605:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716669008346 time 716697118754 state 0x0 ref 1 00000100:00100000:2.0:1789760786.603635:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716697379170 time 716697118754 state 0x1 ref 1 00000100:00100000:2.0:1789760786.604229:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716697379170 time 716728830518 state 0x0 ref 1 00000100:00100000:2.0:1789760786.604256:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503860224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.604329:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79081472, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b70 00010000:00010000:2.0:1789760786.604341:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.604362:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.604367:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.604375:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.604416:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.618071:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986614 is committed 00002000:00010000:2.0:1789760786.618694:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79081472 -> 79085568 00002000:00010000:2.0:1789760786.618702:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154480 -> 154488 00010000:00010000:2.0:1789760786.618720:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.618734:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.618744:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.618757:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.618763:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.618774:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.618800:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.618809:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79081472->79085567] (req 79081472->79085567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.619686:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503860224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15429us (16424us total) trans 4294986614 rc 0/0 00000100:00100000:2.0:1789760786.619701:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19318 00000100:00100000:2.0:1789760786.619714:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716697379170 time 716728830518 state 0x2 ref 0 00000100:00100000:1.0:1789760786.627854:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.628285:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860352 00000100:00100000:2.0:1789760786.628302:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860352 00000100:00100000:2.0:1789760786.628436:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716697379170 time 716728830518 state 0x0 ref 1 00000100:00100000:2.0:1789760786.628455:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716729090934 time 716728830518 state 0x1 ref 1 00000100:00100000:2.0:1789760786.628490:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716729090934 time 716753096695 state 0x0 ref 1 00000100:00100000:2.0:1789760786.628508:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503860352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.628585:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79085568, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b78 00010000:00010000:2.0:1789760786.628601:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.628621:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.628626:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.628634:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.628652:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.637736:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986615 is committed 00002000:00010000:2.0:1789760786.638014:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79085568 -> 79089664 00002000:00010000:2.0:1789760786.638021:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154488 -> 154496 00010000:00010000:2.0:1789760786.638037:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.638051:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.638060:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.638068:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.638075:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.638085:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.638110:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.638119:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f17fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79085568->79089663] (req 79085568->79089663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.638325:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503860352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9812us (10480us total) trans 4294986615 rc 0/0 00000100:00100000:2.0:1789760786.638339:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19319 00000100:00100000:2.0:1789760786.638352:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716729090934 time 716753096695 state 0x2 ref 0 00000100:00100000:1.0:1789760786.642900:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760786.642933:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523612672:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760786.643030:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760786.643152:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612672 00000100:00100000:0.0:1789760786.643162:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612672 00000100:00100000:2.0:1789760786.644246:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:2.0:1789760786.644272:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523612672:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760786.644331:0:7941:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:3.0:1789760786.645001:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.645079:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523612672:12345-0@lo:41:osp-pre-1-0.0 Request processed in 795us (2031us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760786.645094:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:0.0:1789760786.645144:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860480 00000100:00100000:0.0:1789760786.645158:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860480 00000100:00100000:1.0:1789760786.645213:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523612672:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760786.645383:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716729090934 time 716753096695 state 0x0 ref 1 00000100:00100000:0.0:1789760786.650874:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716753357111 time 716753096695 state 0x1 ref 1 00000100:00100000:2.0:1789760786.651239:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716753357111 time 716775841122 state 0x0 ref 1 00000100:00100000:2.0:1789760786.651271:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503860480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.651346:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79089664, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b80 00010000:00010000:2.0:1789760786.651358:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.651378:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.651383:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.651607:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.651627:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.661852:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986616 is committed 00002000:00010000:2.0:1789760786.662119:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79089664 -> 79093760 00002000:00010000:2.0:1789760786.662126:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154496 -> 154504 00010000:00010000:2.0:1789760786.662144:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.662160:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.662172:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.662181:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.662186:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.662203:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.662230:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.662239:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1804 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79089664->79093759] (req 79089664->79093759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.663301:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503860480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12024us (18283us total) trans 4294986616 rc 0/0 00000100:00100000:2.0:1789760786.663318:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19320 00000100:00100000:2.0:1789760786.663336:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716753357111 time 716775841122 state 0x2 ref 0 00000100:00100000:0.0:1789760786.674204:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.674322:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860608 00000100:00100000:3.0:1789760786.674338:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860608 00000100:00100000:3.0:1789760786.674461:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716753357111 time 716775841122 state 0x0 ref 1 00000100:00100000:3.0:1789760786.674480:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716776101538 time 716775841122 state 0x1 ref 1 00000100:00100000:3.0:1789760786.674513:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716776101538 time 716799120279 state 0x0 ref 1 00000100:00100000:3.0:1789760786.674526:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503860608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.674781:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79093760, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b88 00010000:00010000:3.0:1789760786.674794:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.674814:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.674819:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.674827:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.674846:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.683457:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986617 is committed 00002000:00010000:2.0:1789760786.683676:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79093760 -> 79097856 00002000:00010000:2.0:1789760786.683683:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154504 -> 154512 00010000:00010000:2.0:1789760786.683701:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.683717:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.683726:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.683740:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.683746:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.683756:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.683782:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.683791:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f180b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79093760->79097855] (req 79093760->79097855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.683920:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503860608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9387us (9726us total) trans 4294986617 rc 0/0 00000100:00100000:2.0:1789760786.683932:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19321 00000100:00100000:2.0:1789760786.683945:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716776101538 time 716799120279 state 0x2 ref 0 00000100:00100000:1.0:1789760786.690906:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.691067:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860736 00000100:00100000:2.0:1789760786.691081:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860736 00000100:00100000:2.0:1789760786.691153:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716776101538 time 716799120279 state 0x0 ref 1 00000100:00100000:2.0:1789760786.691172:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716799380695 time 716799120279 state 0x1 ref 1 00000100:00100000:2.0:1789760786.691208:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716799380695 time 716815814120 state 0x0 ref 1 00000100:00100000:2.0:1789760786.691225:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503860736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.691441:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79097856, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b90 00010000:00010000:2.0:1789760786.691453:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.691473:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.691479:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.691486:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.691504:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.698998:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986618 is committed 00002000:00010000:2.0:1789760786.699170:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79097856 -> 79101952 00002000:00010000:2.0:1789760786.699177:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154512 -> 154520 00010000:00010000:2.0:1789760786.699194:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.699942:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.699954:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.699963:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.699970:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.699981:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.700006:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.700015:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1812 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79097856->79101951] (req 79097856->79101951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.700252:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503860736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9021us (9353us total) trans 4294986618 rc 0/0 00000100:00100000:2.0:1789760786.700275:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19322 00000100:00100000:2.0:1789760786.700294:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716799380695 time 716815814120 state 0x2 ref 0 00000100:00100000:3.0:1789760786.711154:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.711338:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860864 00000100:00100000:2.0:1789760786.711355:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860864 00000100:00100000:2.0:1789760786.711480:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716799380695 time 716815814120 state 0x0 ref 1 00000100:00100000:2.0:1789760786.711500:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716816074536 time 716815814120 state 0x1 ref 1 00000100:00100000:2.0:1789760786.711549:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716816074536 time 716836155144 state 0x0 ref 1 00000100:00100000:2.0:1789760786.711577:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503860864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.711650:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79101952, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25b98 00010000:00010000:2.0:1789760786.711665:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711686:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.711691:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.711698:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711716:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.717892:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986619 is committed 00002000:00010000:2.0:1789760786.718058:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79101952 -> 79106048 00002000:00010000:2.0:1789760786.718064:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154520 -> 154528 00010000:00010000:2.0:1789760786.718082:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.718097:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.718107:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.718115:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.718121:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.718131:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.718158:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.718167:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1819 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79101952->79106047] (req 79101952->79106047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.718977:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503860864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7404us (7826us total) trans 4294986619 rc 0/0 00000100:00100000:2.0:1789760786.718991:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19323 00000100:00100000:2.0:1789760786.719004:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716816074536 time 716836155144 state 0x2 ref 0 00000100:00100000:0.0:1789760786.726703:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.726787:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503860992 00000100:00100000:2.0:1789760786.726801:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503860992 00000100:00100000:2.0:1789760786.726877:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716816074536 time 716836155144 state 0x0 ref 1 00000100:00100000:2.0:1789760786.726896:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716836415560 time 716836155144 state 0x1 ref 1 00000100:00100000:2.0:1789760786.726988:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716836415560 time 716851592359 state 0x0 ref 1 00000100:00100000:2.0:1789760786.727004:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503860992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.727069:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79106048, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ba0 00010000:00010000:2.0:1789760786.727081:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.727102:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.727108:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.727116:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.727133:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.732364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986620 is committed 00002000:00010000:2.0:1789760786.732770:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79106048 -> 79110144 00002000:00010000:2.0:1789760786.732775:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154528 -> 154536 00010000:00010000:2.0:1789760786.732791:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.732803:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.732810:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.732815:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.732819:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.732827:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.732846:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.732855:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1820 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79106048->79110143] (req 79106048->79110143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.732979:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503860992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5970us (6283us total) trans 4294986620 rc 0/0 00000100:00100000:2.0:1789760786.732991:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19324 00000100:00100000:2.0:1789760786.733004:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716836415560 time 716851592359 state 0x2 ref 0 00000100:00100000:1.0:1789760786.737246:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.737636:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503861120 00000100:00100000:2.0:1789760786.737649:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503861120 00000100:00100000:2.0:1789760786.737719:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716836415560 time 716851592359 state 0x0 ref 1 00000100:00100000:2.0:1789760786.737734:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716851852775 time 716851592359 state 0x1 ref 1 00000100:00100000:2.0:1789760786.737762:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716851852775 time 716862369838 state 0x0 ref 1 00000100:00100000:2.0:1789760786.737773:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503861120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.737825:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79110144, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25ba8 00010000:00010000:2.0:1789760786.737834:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.737850:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.738017:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.738024:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.738041:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.744555:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986621 is committed 00002000:00010000:2.0:1789760786.744774:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79110144 -> 79114240 00002000:00010000:2.0:1789760786.744781:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154536 -> 154544 00010000:00010000:2.0:1789760786.744798:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.744813:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.744823:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.744832:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.744837:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.744847:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.744873:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.744881:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1827 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79110144->79114239] (req 79110144->79114239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.745069:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503861120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7288us (7828us total) trans 4294986621 rc 0/0 00000100:00100000:2.0:1789760786.745083:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19325 00000100:00100000:2.0:1789760786.745095:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716851852775 time 716862369838 state 0x2 ref 0 00000100:00100000:3.0:1789760786.755938:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760786.757445:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503861248 00000100:00100000:0.0:1789760786.757463:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503861248 00000100:00100000:0.0:1789760786.757808:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716851852775 time 716862369838 state 0x0 ref 1 00000100:00100000:0.0:1789760786.757829:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716862630254 time 716862369838 state 0x1 ref 1 00000100:00100000:1.0:1789760786.757958:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716862630254 time 716882562132 state 0x0 ref 1 00000100:00100000:1.0:1789760786.757976:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503861248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.758048:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79114240, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25bb0 00010000:00010000:1.0:1789760786.758060:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.758086:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.758091:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.758098:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.758115:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.763709:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986622 is committed 00002000:00010000:1.0:1789760786.764466:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79114240 -> 79118336 00002000:00010000:1.0:1789760786.764473:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154544 -> 154552 00010000:00010000:1.0:1789760786.764490:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.764505:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.764515:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.764524:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760786.764539:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.764550:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.764580:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.764589:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f182e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79114240->79118335] (req 79114240->79118335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.764808:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503861248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6829us (8879us total) trans 4294986622 rc 0/0 00000100:00100000:1.0:1789760786.764821:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19326 00000100:00100000:1.0:1789760786.764835:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716862630254 time 716882562132 state 0x2 ref 0 00000100:00100000:0.0:1789760786.770708:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.770796:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503861376 00000100:00100000:1.0:1789760786.770810:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503861376 00000100:00100000:1.0:1789760786.770882:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716862630254 time 716882562132 state 0x0 ref 1 00000100:00100000:1.0:1789760786.770900:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716882822548 time 716882562132 state 0x1 ref 1 00000100:00100000:1.0:1789760786.770941:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716882822548 time 716895547576 state 0x0 ref 1 00000100:00100000:1.0:1789760786.770954:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503861376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.771017:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79118336, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25bb8 00010000:00010000:1.0:1789760786.771028:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.771049:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.771055:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.771062:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.771078:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.776592:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986623 is committed 00002000:00010000:3.0:1789760786.776877:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79118336 -> 79122432 00002000:00010000:3.0:1789760786.776884:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154552 -> 154560 00010000:00010000:3.0:1789760786.776901:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.776917:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.776937:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.776946:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.776952:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.776963:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.776988:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.776997:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1835 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79118336->79122431] (req 79118336->79122431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.777136:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503861376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6176us (6436us total) trans 4294986623 rc 0/0 00000100:00100000:3.0:1789760786.777150:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19327 00000100:00100000:3.0:1789760786.777163:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716882822548 time 716895547576 state 0x2 ref 0 00000100:00100000:0.0:1789760786.783838:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.784211:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503861504 00000100:00100000:2.0:1789760786.784224:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503861504 00000100:00100000:2.0:1789760786.784305:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716882822548 time 716895547576 state 0x0 ref 1 00000100:00100000:2.0:1789760786.784324:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716895807992 time 716895547576 state 0x1 ref 1 00000100:00100000:2.0:1789760786.784362:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716895807992 time 716908968044 state 0x0 ref 1 00000100:00100000:2.0:1789760786.784376:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503861504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.784479:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79122432, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x25bc0 00010000:00010000:2.0:1789760786.784490:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.784510:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760786.784516:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.784541:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.784558:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.792507:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986624 is committed 00002000:00010000:2.0:1789760786.792742:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79122432 -> 79126528 00002000:00010000:2.0:1789760786.792748:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:2.0:1789760786.792750:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00002000:00010000:2.0:1789760786.792753:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154560 -> 154568 00010000:00010000:2.0:1789760786.792770:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.792787:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.792797:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.792804:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.792810:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.792821:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.792846:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.792854:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f183c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79122432->79126527] (req 79122432->79126527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.793072:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503861504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8690us (9241us total) trans 4294986624 rc 0/0 00000100:00100000:2.0:1789760786.793086:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19328 00000100:00100000:2.0:1789760786.793099:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716895807992 time 716908968044 state 0x2 ref 0 00000100:00100000:1.0:1789760786.798376:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760786.798807:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503861632 00000100:00100000:0.0:1789760786.798822:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503861632 00000100:00100000:0.0:1789760786.798899:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716895807992 time 716908968044 state 0x0 ref 1 00000100:00100000:0.0:1789760786.798918:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716909228460 time 716908968044 state 0x1 ref 1 00000100:00100000:0.0:1789760786.798994:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716909228460 time 716923599609 state 0x0 ref 1 00000100:00100000:0.0:1789760786.799010:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503861632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760786.799078:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79126528, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25bc8 00010000:00010000:0.0:1789760786.799091:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.799112:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760786.799118:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.799127:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.799144:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.807383:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986625 is committed 00002000:00010000:0.0:1789760786.807621:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79126528 -> 79130624 00002000:00010000:0.0:1789760786.807628:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154568 -> 154576 00010000:00010000:0.0:1789760786.807647:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.807662:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.807678:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.807687:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760786.807693:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.807703:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.807728:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.807736:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1843 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79126528->79130623] (req 79126528->79130623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.807913:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503861632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8890us (9650us total) trans 4294986625 rc 0/0 00000100:00100000:0.0:1789760786.807925:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19329 00000100:00100000:0.0:1789760786.807938:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716909228460 time 716923599609 state 0x2 ref 0 00000100:00100000:2.0:1789760786.816121:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760786.816239:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503861760 00000100:00100000:1.0:1789760786.816253:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503861760 00000100:00100000:1.0:1789760786.816340:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716909228460 time 716923599609 state 0x0 ref 1 00000100:00100000:1.0:1789760786.816360:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716923860025 time 716923599609 state 0x1 ref 1 00000100:00100000:1.0:1789760786.816494:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716923860025 time 716941099325 state 0x0 ref 1 00000100:00100000:1.0:1789760786.816510:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503861760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760786.816757:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79130624, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25bd0 00010000:00010000:1.0:1789760786.816770:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.816791:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760786.816796:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.816804:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.816823:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.826341:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986626 is committed 00002000:00010000:2.0:1789760786.826792:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79130624 -> 79134720 00002000:00010000:2.0:1789760786.826799:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154576 -> 154584 00010000:00010000:2.0:1789760786.826819:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.826836:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.826846:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.826854:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.826860:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.826871:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.826895:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.826905:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f184a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79130624->79134719] (req 79130624->79134719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.827108:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503861760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10593us (10993us total) trans 4294986626 rc 0/0 00000100:00100000:2.0:1789760786.827122:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19330 00000100:00100000:2.0:1789760786.827135:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716923860025 time 716941099325 state 0x2 ref 0 00000100:00100000:1.0:1789760786.841028:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.841371:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503861888 00000100:00100000:3.0:1789760786.841385:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503861888 00000100:00100000:3.0:1789760786.841543:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716923860025 time 716941099325 state 0x0 ref 1 00000100:00100000:3.0:1789760786.841564:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716941359741 time 716941099325 state 0x1 ref 1 00000100:00100000:3.0:1789760786.841593:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716941359741 time 716966198456 state 0x0 ref 1 00000100:00100000:3.0:1789760786.841608:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503861888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.841675:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79134720, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25bd8 00010000:00010000:3.0:1789760786.841687:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.842226:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760786.842236:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.842244:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.842267:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.849349:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986627 is committed 00002000:00010000:3.0:1789760786.853259:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79134720 -> 79138816 00002000:00010000:3.0:1789760786.853275:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154584 -> 154592 00010000:00010000:3.0:1789760786.853294:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853310:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853320:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853328:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.853333:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853344:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853371:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853380:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1851 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79134720->79138815] (req 79134720->79138815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.853558:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503861888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11945us (12537us total) trans 4294986627 rc 0/0 00000100:00100000:3.0:1789760786.853572:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19331 00000100:00100000:3.0:1789760786.853585:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716941359741 time 716966198456 state 0x2 ref 0 00000100:00100000:2.0:1789760786.859750:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760786.859883:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862016 00000100:00100000:0.0:1789760786.859899:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862016 00000100:00100000:0.0:1789760786.859976:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716941359741 time 716966198456 state 0x0 ref 1 00000100:00100000:0.0:1789760786.859996:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716966458872 time 716966198456 state 0x1 ref 1 00000100:00100000:0.0:1789760786.860034:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716966458872 time 716984639793 state 0x0 ref 1 00000100:00100000:0.0:1789760786.860048:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc14050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503862016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760786.860117:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79138816, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25be0 00010000:00010000:0.0:1789760786.860128:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.860148:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760786.860154:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.860162:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.860179:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.869652:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986628 is committed 00002000:00010000:2.0:1789760786.869893:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79138816 -> 79142912 00002000:00010000:2.0:1789760786.869899:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154592 -> 154600 00010000:00010000:2.0:1789760786.869916:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.869932:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.869942:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.869950:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.869956:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.869967:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.869996:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.870005:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1858 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79138816->79142911] (req 79138816->79142911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.870143:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc14050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503862016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10089us (10400us total) trans 4294986628 rc 0/0 00000100:00100000:2.0:1789760786.870156:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19332 00000100:00100000:2.0:1789760786.870170:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716966458872 time 716984639793 state 0x2 ref 0 00000100:00100000:2.0:1789760786.876882:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760786.877477:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862144 00000100:00100000:0.0:1789760786.877492:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862144 00000100:00100000:0.0:1789760786.877588:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716966458872 time 716984639793 state 0x0 ref 1 00000100:00100000:0.0:1789760786.877607:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716984900209 time 716984639793 state 0x1 ref 1 00000100:00100000:0.0:1789760786.877703:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716984900209 time 717002307306 state 0x0 ref 1 00000100:00100000:0.0:1789760786.877719:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503862144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760786.877788:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79142912, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25be8 00010000:00010000:0.0:1789760786.877800:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.877822:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760786.877828:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.877836:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.877856:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.883961:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986629 is committed 00002000:00010000:0.0:1789760786.884090:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79142912 -> 79147008 00002000:00010000:0.0:1789760786.884097:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154600 -> 154608 00010000:00010000:0.0:1789760786.884115:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.884131:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.884141:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.884150:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760786.884156:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.884174:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.884199:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.884208:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f185f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79142912->79147007] (req 79142912->79147007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.884955:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503862144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7230us (8079us total) trans 4294986629 rc 0/0 00000100:00100000:0.0:1789760786.884970:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19333 00000100:00100000:0.0:1789760786.884984:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716984900209 time 717002307306 state 0x2 ref 0 00000100:00100000:1.0:1789760786.894347:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.895060:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862272 00000100:00100000:2.0:1789760786.895076:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862272 00000100:00100000:2.0:1789760786.895330:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 716984900209 time 717002307306 state 0x0 ref 1 00000100:00100000:2.0:1789760786.895351:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717002567722 time 717002307306 state 0x1 ref 1 00000100:00100000:2.0:1789760786.895437:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717002567722 time 717019995841 state 0x0 ref 1 00000100:00100000:2.0:1789760786.895453:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.895520:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79147008, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25bf0 00010000:00010000:2.0:1789760786.895541:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.895568:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760786.895575:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.895583:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.895601:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.915345:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986630 is committed 00002000:00010000:3.0:1789760786.915690:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79147008 -> 79151104 00002000:00010000:3.0:1789760786.915697:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154608 -> 154616 00010000:00010000:3.0:1789760786.915724:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.915739:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.915749:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.915757:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.915763:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.915773:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.915798:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.915913:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1866 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79147008->79151103] (req 79147008->79151103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.916615:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 21157us (22276us total) trans 4294986630 rc 0/0 00000100:00100000:3.0:1789760786.916630:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19334 00000100:00100000:3.0:1789760786.916643:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717002567722 time 717019995841 state 0x2 ref 0 00000100:00100000:2.0:1789760786.925741:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.925960:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862400 00000100:00100000:3.0:1789760786.925974:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862400 00000100:00100000:3.0:1789760786.926052:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717002567722 time 717019995841 state 0x0 ref 1 00000100:00100000:3.0:1789760786.926072:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717020256257 time 717019995841 state 0x1 ref 1 00000100:00100000:3.0:1789760786.926100:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717020256257 time 717050706411 state 0x0 ref 1 00000100:00100000:3.0:1789760786.926114:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.926183:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79151104, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25bf8 00010000:00010000:3.0:1789760786.926195:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.926215:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760786.926220:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.926227:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.926243:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.935449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986631 is committed 00002000:00010000:3.0:1789760786.936212:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79151104 -> 79155200 00002000:00010000:3.0:1789760786.936219:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154616 -> 154624 00010000:00010000:3.0:1789760786.936238:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.936314:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.936327:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.936336:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.936342:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.936353:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.936382:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.936437:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f186d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79151104->79155199] (req 79151104->79155199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.936696:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10577us (10964us total) trans 4294986631 rc 0/0 00000100:00100000:3.0:1789760786.936720:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19335 00000100:00100000:3.0:1789760786.936733:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717020256257 time 717050706411 state 0x2 ref 0 00000100:00100000:2.0:1789760786.946298:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760786.946380:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862528 00000100:00100000:0.0:1789760786.946448:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862528 00000100:00100000:0.0:1789760786.946536:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717020256257 time 717050706411 state 0x0 ref 1 00000100:00100000:0.0:1789760786.946556:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717050966827 time 717050706411 state 0x1 ref 1 00000100:00100000:0.0:1789760786.946593:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717050966827 time 717071198648 state 0x0 ref 1 00000100:00100000:0.0:1789760786.946608:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503862528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760786.946675:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79155200, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c00 00010000:00010000:0.0:1789760786.946686:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.946706:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760786.946711:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.946718:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.946734:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.953024:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986632 is committed 00002000:00010000:2.0:1789760786.953852:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79155200 -> 79159296 00002000:00010000:2.0:1789760786.953862:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154624 -> 154632 00010000:00010000:2.0:1789760786.953880:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.953896:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.953905:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.953914:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.953920:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.953931:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.953957:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.953966:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1874 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79155200->79159295] (req 79155200->79159295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.954101:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503862528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7488us (7813us total) trans 4294986632 rc 0/0 00000100:00100000:2.0:1789760786.954114:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19336 00000100:00100000:2.0:1789760786.954128:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717050966827 time 717071198648 state 0x2 ref 0 00000100:00100000:0.0:1789760786.962109:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760786.962247:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862656 00000100:00100000:2.0:1789760786.962692:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862656 00000100:00100000:2.0:1789760786.962778:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717050966827 time 717071198648 state 0x0 ref 1 00000100:00100000:2.0:1789760786.962798:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717071459064 time 717071198648 state 0x1 ref 1 00000100:00100000:2.0:1789760786.962905:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717071459064 time 717087509966 state 0x0 ref 1 00000100:00100000:2.0:1789760786.962922:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760786.962995:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79159296, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c08 00010000:00010000:2.0:1789760786.963015:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.963038:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760786.963044:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.963051:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.963070:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.963646:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760786.971080:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986633 is committed 00002000:00010000:2.0:1789760786.971918:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79159296 -> 79163392 00002000:00010000:2.0:1789760786.971928:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154632 -> 154640 00010000:00010000:2.0:1789760786.971945:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.971962:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.971972:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.971980:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760786.971986:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.971995:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.972020:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.972034:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f187b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79159296->79163391] (req 79159296->79163391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.972171:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9244us (10070us total) trans 4294986633 rc 0/0 00000100:00100000:2.0:1789760786.972183:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19337 00000100:00100000:2.0:1789760786.972197:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717071459064 time 717087509966 state 0x2 ref 0 00000100:00100000:1.0:1789760786.986946:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760786.987188:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862784 00000100:00100000:3.0:1789760786.987202:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862784 00000100:00100000:3.0:1789760786.987279:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717071459064 time 717087509966 state 0x0 ref 1 00000100:00100000:3.0:1789760786.987298:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717087770382 time 717087509966 state 0x1 ref 1 00000100:00100000:3.0:1789760786.987475:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717087770382 time 717112079375 state 0x0 ref 1 00000100:00100000:3.0:1789760786.987490:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760786.987573:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79163392, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c10 00010000:00010000:3.0:1789760786.987585:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.987604:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760786.987609:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.987617:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.987635:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.995519:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986634 is committed 00002000:00010000:3.0:1789760786.996228:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79163392 -> 79167488 00002000:00010000:3.0:1789760786.996236:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154640 -> 154648 00010000:00010000:3.0:1789760786.996255:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.996341:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.996350:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.996358:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760786.996364:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.996375:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.996428:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.996438:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1882 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79163392->79167487] (req 79163392->79167487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.996597:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503862784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9102us (9658us total) trans 4294986634 rc 0/0 00000100:00100000:3.0:1789760786.996611:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19338 00000100:00100000:3.0:1789760786.996624:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717087770382 time 717112079375 state 0x2 ref 0 00000100:00100000:0.0:1789760787.002584:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.002900:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503862912 00000100:00100000:3.0:1789760787.002913:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503862912 00000100:00100000:3.0:1789760787.002986:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717087770382 time 717112079375 state 0x0 ref 1 00000100:00100000:3.0:1789760787.003474:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717112339791 time 717112079375 state 0x1 ref 1 00000100:00100000:3.0:1789760787.003775:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717112339791 time 717128378431 state 0x0 ref 1 00000100:00100000:3.0:1789760787.003795:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503862912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.003863:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79167488, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c18 00010000:00010000:3.0:1789760787.003875:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.003896:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.003900:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.003908:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.003925:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.010345:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986635 is committed 00002000:00010000:3.0:1789760787.010807:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79167488 -> 79171584 00002000:00010000:3.0:1789760787.010814:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154648 -> 154656 00010000:00010000:3.0:1789760787.010832:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.010854:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.010863:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.010871:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.010876:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.010886:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.010911:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.010918:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1889 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79167488->79171583] (req 79167488->79171583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.011790:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503862912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7989us (9208us total) trans 4294986635 rc 0/0 00000100:00100000:3.0:1789760787.011806:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19339 00000100:00100000:3.0:1789760787.011819:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717112339791 time 717128378431 state 0x2 ref 0 00000100:00100000:0.0:1789760787.018253:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.018442:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863040 00000100:00100000:2.0:1789760787.018457:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863040 00000100:00100000:2.0:1789760787.018544:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717112339791 time 717128378431 state 0x0 ref 1 00000100:00100000:2.0:1789760787.019629:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717128638847 time 717128378431 state 0x1 ref 1 00000100:00100000:2.0:1789760787.019699:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717128638847 time 717144301731 state 0x0 ref 1 00000100:00100000:2.0:1789760787.019722:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503863040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.019796:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79171584, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c20 00010000:00010000:2.0:1789760787.019808:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.019829:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.019834:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.019841:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.019864:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.029973:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986636 is committed 00002000:00010000:2.0:1789760787.030129:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79171584 -> 79175680 00002000:00010000:2.0:1789760787.030136:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154656 -> 154664 00010000:00010000:2.0:1789760787.030153:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.030168:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.030177:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.030186:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.030192:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.030203:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.030232:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.030240:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1890 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79171584->79175679] (req 79171584->79175679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.030386:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503863040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10664us (12140us total) trans 4294986636 rc 0/0 00000100:00100000:2.0:1789760787.030444:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19340 00000100:00100000:2.0:1789760787.030457:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717128638847 time 717144301731 state 0x2 ref 0 00000100:00100000:2.0:1789760787.034949:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.035090:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863168 00000100:00100000:0.0:1789760787.035108:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863168 00000100:00100000:0.0:1789760787.035188:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717128638847 time 717144301731 state 0x0 ref 1 00000100:00100000:0.0:1789760787.035214:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717144562147 time 717144301731 state 0x1 ref 1 00000100:00100000:0.0:1789760787.035250:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717144562147 time 717159856436 state 0x0 ref 1 00000100:00100000:0.0:1789760787.035264:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503863168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.035340:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79175680, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c28 00010000:00010000:0.0:1789760787.035352:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.035373:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.035379:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.035386:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.035447:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.044103:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986637 is committed 00002000:00010000:2.0:1789760787.044269:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79175680 -> 79179776 00002000:00010000:2.0:1789760787.044275:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154664 -> 154672 00010000:00010000:2.0:1789760787.044293:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044307:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044383:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044424:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.044429:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044439:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044474:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044483:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1897 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79175680->79179775] (req 79175680->79179775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.044603:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503863168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9335us (9662us total) trans 4294986637 rc 0/0 00000100:00100000:2.0:1789760787.044615:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19341 00000100:00100000:2.0:1789760787.044628:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717144562147 time 717159856436 state 0x2 ref 0 00000100:00100000:0.0:1789760787.050705:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.051688:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863296 00000100:00100000:3.0:1789760787.051706:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863296 00000100:00100000:3.0:1789760787.051795:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717144562147 time 717159856436 state 0x0 ref 1 00000100:00100000:3.0:1789760787.051816:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717160116852 time 717159856436 state 0x1 ref 1 00000100:00100000:3.0:1789760787.051855:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717160116852 time 717176461554 state 0x0 ref 1 00000100:00100000:3.0:1789760787.051871:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503863296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.051938:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79179776, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c30 00010000:00010000:3.0:1789760787.051950:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.051972:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.051977:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.051985:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.052002:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.062236:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986638 is committed 00002000:00010000:3.0:1789760787.062764:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79179776 -> 79183872 00002000:00010000:3.0:1789760787.062772:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154672 -> 154680 00010000:00010000:3.0:1789760787.062789:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.062802:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.062816:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.062824:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.062830:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.062841:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.062867:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.062875:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f189e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79179776->79183871] (req 79179776->79183871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.063087:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503863296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11213us (12390us total) trans 4294986638 rc 0/0 00000100:00100000:3.0:1789760787.064352:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19342 00000100:00100000:3.0:1789760787.064373:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717160116852 time 717176461554 state 0x2 ref 0 00000100:00100000:0.0:1789760787.075519:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.075864:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863424 00000100:00100000:3.0:1789760787.075878:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863424 00000100:00100000:3.0:1789760787.076003:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717160116852 time 717176461554 state 0x0 ref 1 00000100:00100000:3.0:1789760787.076022:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717176721970 time 717176461554 state 0x1 ref 1 00000100:00100000:3.0:1789760787.076068:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717176721970 time 717200673747 state 0x0 ref 1 00000100:00100000:3.0:1789760787.076085:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503863424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.076151:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79183872, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c38 00010000:00010000:3.0:1789760787.076162:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.076183:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.076189:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.076198:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.076216:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.086129:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986639 is committed 00002000:00010000:3.0:1789760787.086348:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79183872 -> 79187968 00002000:00010000:3.0:1789760787.086355:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154680 -> 154688 00010000:00010000:3.0:1789760787.086372:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.086458:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.086471:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.086485:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.086491:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.086502:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.086538:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.086547:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79183872->79187967] (req 79183872->79187967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.087051:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503863424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10959us (11538us total) trans 4294986639 rc 0/0 00000100:00100000:3.0:1789760787.087065:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19343 00000100:00100000:3.0:1789760787.087078:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717176721970 time 717200673747 state 0x2 ref 0 00000100:00100000:2.0:1789760787.093603:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.093879:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863552 00000100:00100000:0.0:1789760787.093904:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863552 00000100:00100000:0.0:1789760787.093981:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717176721970 time 717200673747 state 0x0 ref 1 00000100:00100000:0.0:1789760787.094000:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717200934163 time 717200673747 state 0x1 ref 1 00000100:00100000:0.0:1789760787.094038:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717200934163 time 717218643658 state 0x0 ref 1 00000100:00100000:0.0:1789760787.094052:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.094116:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79187968, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c40 00010000:00010000:0.0:1789760787.094129:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.094151:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.094157:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.094165:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.094181:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.111804:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986640 is committed 00002000:00010000:0.0:1789760787.111967:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79187968 -> 79192064 00002000:00010000:0.0:1789760787.111975:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154688 -> 154696 00010000:00010000:0.0:1789760787.111994:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.112010:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.112020:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.112029:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.112034:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.112045:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.112071:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.112086:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79187968->79192063] (req 79187968->79192063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.112273:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18214us (18676us total) trans 4294986640 rc 0/0 00000100:00100000:0.0:1789760787.112298:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19344 00000100:00100000:0.0:1789760787.112312:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717200934163 time 717218643658 state 0x2 ref 0 00000100:00100000:0.0:1789760787.120608:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.120875:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863680 00000100:00100000:2.0:1789760787.120890:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863680 00000100:00100000:2.0:1789760787.120968:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717200934163 time 717218643658 state 0x0 ref 1 00000100:00100000:2.0:1789760787.120987:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717218904074 time 717218643658 state 0x1 ref 1 00000100:00100000:2.0:1789760787.121027:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717218904074 time 717245632643 state 0x0 ref 1 00000100:00100000:2.0:1789760787.121040:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.121107:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79192064, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c48 00010000:00010000:2.0:1789760787.121118:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.121140:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.121146:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.121154:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.121170:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.130681:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986641 is committed 00002000:00010000:2.0:1789760787.130996:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79192064 -> 79196160 00002000:00010000:2.0:1789760787.131004:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154696 -> 154704 00010000:00010000:2.0:1789760787.131024:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.131042:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.131052:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.131059:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.131065:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.131075:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.131100:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.131115:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f18b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79192064->79196159] (req 79192064->79196159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.131263:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10216us (10661us total) trans 4294986641 rc 0/0 00000100:00100000:2.0:1789760787.131280:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19345 00000100:00100000:2.0:1789760787.131306:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717218904074 time 717245632643 state 0x2 ref 0 00000100:00100000:2.0:1789760787.141016:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.141200:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863808 00000100:00100000:3.0:1789760787.141215:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863808 00000100:00100000:3.0:1789760787.141298:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717218904074 time 717245632643 state 0x0 ref 1 00000100:00100000:3.0:1789760787.141318:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717245893059 time 717245632643 state 0x1 ref 1 00000100:00100000:3.0:1789760787.141354:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717245893059 time 717265959451 state 0x0 ref 1 00000100:00100000:3.0:1789760787.141372:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.141463:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79196160, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c50 00010000:00010000:3.0:1789760787.141475:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.141496:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.141502:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.141509:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.141535:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.153590:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986642 is committed 00002000:00010000:3.0:1789760787.154337:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79196160 -> 79200256 00002000:00010000:3.0:1789760787.154347:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154704 -> 154712 00010000:00010000:3.0:1789760787.154373:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.154448:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.154459:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.154468:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.154476:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.154487:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.154525:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.154541:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79196160->79200255] (req 79196160->79200255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.154686:0:3637:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760787.154707:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13329us (13697us total) trans 4294986642 rc 0/0 00000100:00100000:0.0:1789760787.154711:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5ccdfdb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523612800:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760787.154719:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19346 00000100:00100000:3.0:1789760787.154731:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717245893059 time 717265959451 state 0x2 ref 0 00000100:00100000:0.0:1789760787.154783:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.154911:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612800 00000100:00100000:3.0:1789760787.154920:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612800 00000100:00100000:0.0:1789760787.155126:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:0.0:1789760787.155144:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523612800:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760787.155194:0:6517:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:0.0:1789760787.155264:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523612800:12345-0@lo:41:osp-pre-0-1.0 Request processed in 129us (481us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.155271:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:2.0:1789760787.155441:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5ccdfdb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523612800:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760787.162843:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.163166:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503863936 00000100:00100000:3.0:1789760787.163179:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503863936 00000100:00100000:3.0:1789760787.163262:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717245893059 time 717265959451 state 0x0 ref 1 00000100:00100000:3.0:1789760787.163282:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717266219867 time 717265959451 state 0x1 ref 1 00000100:00100000:3.0:1789760787.163314:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717266219867 time 717287921023 state 0x0 ref 1 00000100:00100000:3.0:1789760787.163328:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.163425:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79200256, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c58 00010000:00010000:3.0:1789760787.163436:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.163454:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.163460:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.163467:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.163483:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.172775:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986643 is committed 00002000:00010000:3.0:1789760787.173509:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79200256 -> 79204352 00002000:00010000:3.0:1789760787.173516:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154712 -> 154720 00010000:00010000:3.0:1789760787.173541:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.173558:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.173576:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.173585:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.173591:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.173609:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.173633:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.173641:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79200256->79204351] (req 79200256->79204351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.173782:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503863936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10450us (10946us total) trans 4294986643 rc 0/0 00000100:00100000:3.0:1789760787.173795:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19347 00000100:00100000:3.0:1789760787.173806:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717266219867 time 717287921023 state 0x2 ref 0 00000100:00100000:1.0:1789760787.180109:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.180213:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864064 00000100:00100000:0.0:1789760787.180227:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864064 00000100:00100000:0.0:1789760787.180309:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717266219867 time 717287921023 state 0x0 ref 1 00000100:00100000:0.0:1789760787.180328:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717288181439 time 717287921023 state 0x1 ref 1 00000100:00100000:0.0:1789760787.180360:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717288181439 time 717304966077 state 0x0 ref 1 00000100:00100000:0.0:1789760787.180373:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.180474:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79204352, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c60 00010000:00010000:0.0:1789760787.180485:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.180506:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.180511:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.180520:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.180550:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.188922:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986644 is committed 00002000:00010000:0.0:1789760787.189233:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79204352 -> 79208448 00002000:00010000:0.0:1789760787.189239:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154720 -> 154728 00010000:00010000:0.0:1789760787.190225:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.190246:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.190265:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.190281:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.190286:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.190297:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.190322:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.190329:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f18c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79204352->79208447] (req 79204352->79208447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.190502:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10123us (10398us total) trans 4294986644 rc 0/0 00000100:00100000:0.0:1789760787.190514:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19348 00000100:00100000:0.0:1789760787.190536:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717288181439 time 717304966077 state 0x2 ref 0 00000100:00100000:2.0:1789760787.202715:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.202830:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864192 00000100:00100000:1.0:1789760787.202842:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864192 00000100:00100000:1.0:1789760787.202911:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717288181439 time 717304966077 state 0x0 ref 1 00000100:00100000:1.0:1789760787.202929:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717305226493 time 717304966077 state 0x1 ref 1 00000100:00100000:1.0:1789760787.202964:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717305226493 time 717327570119 state 0x0 ref 1 00000100:00100000:1.0:1789760787.202976:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.203046:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79208448, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c68 00010000:00010000:1.0:1789760787.203057:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.203076:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.203082:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.203089:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.203106:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.213709:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986645 is committed 00002000:00010000:1.0:1789760787.222331:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79208448 -> 79212544 00002000:00010000:1.0:1789760787.222346:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154728 -> 154736 00010000:00010000:1.0:1789760787.222365:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.222383:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.222410:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.222419:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.222426:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.222436:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.222462:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.222470:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f18cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79208448->79212543] (req 79208448->79212543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.222789:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19805us (20079us total) trans 4294986645 rc 0/0 00000100:00100000:1.0:1789760787.222803:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19349 00000100:00100000:1.0:1789760787.222816:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717305226493 time 717327570119 state 0x2 ref 0 00000100:00100000:1.0:1789760787.236420:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.236610:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864320 00000100:00100000:3.0:1789760787.236623:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864320 00000100:00100000:3.0:1789760787.236698:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717305226493 time 717327570119 state 0x0 ref 1 00000100:00100000:3.0:1789760787.236719:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717327830535 time 717327570119 state 0x1 ref 1 00000100:00100000:3.0:1789760787.236760:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717327830535 time 717361365613 state 0x0 ref 1 00000100:00100000:3.0:1789760787.236775:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.236838:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79212544, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c70 00010000:00010000:3.0:1789760787.236851:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.236872:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.236877:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.236886:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.236904:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.245296:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986646 is committed 00002000:00010000:3.0:1789760787.245468:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79212544 -> 79216640 00002000:00010000:3.0:1789760787.245475:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154736 -> 154744 00010000:00010000:3.0:1789760787.245492:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.245509:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.245520:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.245541:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.245547:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.245557:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.245642:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.245652:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79212544->79216639] (req 79212544->79216639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.245806:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9024us (9414us total) trans 4294986646 rc 0/0 00000100:00100000:3.0:1789760787.245818:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19350 00000100:00100000:3.0:1789760787.245830:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717327830535 time 717361365613 state 0x2 ref 0 00000100:00100000:0.0:1789760787.250204:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.250321:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864448 00000100:00100000:3.0:1789760787.250336:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864448 00000100:00100000:3.0:1789760787.250441:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717327830535 time 717361365613 state 0x0 ref 1 00000100:00100000:3.0:1789760787.250462:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717361626029 time 717361365613 state 0x1 ref 1 00000100:00100000:2.0:1789760787.250581:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717361626029 time 717375184247 state 0x0 ref 1 00000100:00100000:2.0:1789760787.250598:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503864448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.250747:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79216640, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c78 00010000:00010000:2.0:1789760787.250759:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.250780:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.250785:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.250795:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.250818:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.257765:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986647 is committed 00002000:00010000:1.0:1789760787.258246:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79216640 -> 79220736 00002000:00010000:1.0:1789760787.258254:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154744 -> 154752 00010000:00010000:1.0:1789760787.258273:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.258288:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.258298:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.258306:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.258312:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.258324:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.258578:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.258589:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f18dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79216640->79220735] (req 79216640->79220735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.258912:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503864448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8301us (8709us total) trans 4294986647 rc 0/0 00000100:00100000:1.0:1789760787.258934:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19351 00000100:00100000:1.0:1789760787.258948:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717361626029 time 717375184247 state 0x2 ref 0 00000100:00100000:2.0:1789760787.264260:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.264442:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864576 00000100:00100000:0.0:1789760787.264457:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864576 00000100:00100000:0.0:1789760787.264548:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717361626029 time 717375184247 state 0x0 ref 1 00000100:00100000:0.0:1789760787.264937:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717375444663 time 717375184247 state 0x1 ref 1 00000100:00100000:0.0:1789760787.265020:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717375444663 time 717389626213 state 0x0 ref 1 00000100:00100000:0.0:1789760787.265038:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.265104:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79220736, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c80 00010000:00010000:0.0:1789760787.265116:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.265137:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.265142:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.265151:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.265167:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.271546:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986648 is committed 00002000:00010000:0.0:1789760787.271824:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79220736 -> 79224832 00002000:00010000:0.0:1789760787.271831:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154752 -> 154760 00010000:00010000:0.0:1789760787.271849:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.271864:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.271874:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.271888:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.271894:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.271904:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.271928:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.271937:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f18e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79220736->79224831] (req 79220736->79224831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.272064:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7023us (7811us total) trans 4294986648 rc 0/0 00000100:00100000:0.0:1789760787.272076:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19352 00000100:00100000:0.0:1789760787.272089:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717375444663 time 717389626213 state 0x2 ref 0 00000100:00100000:2.0:1789760787.281050:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.286747:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864704 00000100:00100000:1.0:1789760787.286770:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864704 00000100:00100000:1.0:1789760787.286866:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717375444663 time 717389626213 state 0x0 ref 1 00000100:00100000:1.0:1789760787.286887:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717389886629 time 717389626213 state 0x1 ref 1 00000100:00100000:1.0:1789760787.287001:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717389886629 time 717411606341 state 0x0 ref 1 00000100:00100000:1.0:1789760787.287019:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503864704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.287092:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79224832, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c88 00010000:00010000:1.0:1789760787.287106:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.287127:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.287133:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.287141:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.287159:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.294317:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986649 is committed 00002000:00010000:1.0:1789760787.294481:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79224832 -> 79228928 00002000:00010000:1.0:1789760787.294488:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154760 -> 154768 00010000:00010000:1.0:1789760787.294506:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.294521:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.294542:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.294562:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.294569:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.294580:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.294612:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.294621:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79224832->79228927] (req 79224832->79228927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.294761:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503864704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7738us (13716us total) trans 4294986649 rc 0/0 00000100:00100000:1.0:1789760787.294774:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19353 00000100:00100000:1.0:1789760787.294788:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717389886629 time 717411606341 state 0x2 ref 0 00000100:00100000:1.0:1789760787.311228:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.311486:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864832 00000100:00100000:3.0:1789760787.311500:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864832 00000100:00100000:3.0:1789760787.311602:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717389886629 time 717411606341 state 0x0 ref 1 00000100:00100000:3.0:1789760787.311622:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717411866757 time 717411606341 state 0x1 ref 1 00000100:00100000:3.0:1789760787.311657:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717411866757 time 717436263207 state 0x0 ref 1 00000100:00100000:3.0:1789760787.311670:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.311742:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79228928, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c90 00010000:00010000:3.0:1789760787.311754:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.311774:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.311779:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.311787:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.311806:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.324112:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986650 is committed 00002000:00010000:2.0:1789760787.324335:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79228928 -> 79233024 00002000:00010000:2.0:1789760787.324342:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154768 -> 154776 00010000:00010000:2.0:1789760787.324361:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.324377:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.324387:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.324430:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.324436:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.324446:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.324474:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.324483:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f18f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79228928->79233023] (req 79228928->79233023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.330771:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503864832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19079us (19536us total) trans 4294986650 rc 0/0 00000100:00100000:0.0:1789760787.330791:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19354 00000100:00100000:0.0:1789760787.330805:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717411866757 time 717436263207 state 0x2 ref 0 00000100:00100000:2.0:1789760787.339086:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.341599:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503864960 00000100:00100000:1.0:1789760787.341621:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503864960 00000100:00100000:1.0:1789760787.341713:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717411866757 time 717436263207 state 0x0 ref 1 00000100:00100000:1.0:1789760787.341733:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717436523623 time 717436263207 state 0x1 ref 1 00000100:00100000:1.0:1789760787.341777:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717436523623 time 717466383336 state 0x0 ref 1 00000100:00100000:1.0:1789760787.341793:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503864960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.341856:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79233024, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25c98 00010000:00010000:1.0:1789760787.341867:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.341893:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.341898:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.341918:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.341935:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.351457:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986651 is committed 00002000:00010000:1.0:1789760787.351589:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79233024 -> 79237120 00002000:00010000:1.0:1789760787.351595:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154776 -> 154784 00010000:00010000:1.0:1789760787.351612:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.351627:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.351637:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.351645:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.351650:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.351660:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.351691:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.351700:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f18f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79233024->79237119] (req 79233024->79237119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.351824:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503864960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10028us (12749us total) trans 4294986651 rc 0/0 00000100:00100000:1.0:1789760787.351836:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19355 00000100:00100000:1.0:1789760787.351848:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717436523623 time 717466383336 state 0x2 ref 0 00000100:00100000:2.0:1789760787.370731:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.370804:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865088 00000100:00100000:0.0:1789760787.370816:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865088 00000100:00100000:0.0:1789760787.370890:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717436523623 time 717466383336 state 0x0 ref 1 00000100:00100000:0.0:1789760787.371343:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717466643752 time 717466383336 state 0x1 ref 1 00000100:00100000:0.0:1789760787.371519:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717466643752 time 717496123349 state 0x0 ref 1 00000100:00100000:0.0:1789760787.371549:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.371630:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79237120, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25ca0 00010000:00010000:0.0:1789760787.371642:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.371662:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.371667:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.371674:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.371691:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.384558:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986652 is committed 00002000:00010000:0.0:1789760787.395622:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79237120 -> 79241216 00002000:00010000:0.0:1789760787.395633:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154784 -> 154792 00010000:00010000:0.0:1789760787.395647:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.395667:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.395676:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.395684:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.395689:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.395699:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.395717:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.395737:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1900 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79237120->79241215] (req 79237120->79241215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.395849:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 24302us (25129us total) trans 4294986652 rc 0/0 00000100:00100000:0.0:1789760787.395860:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19356 00000100:00100000:0.0:1789760787.395871:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717466643752 time 717496123349 state 0x2 ref 0 00000100:00100000:0.0:1789760787.402540:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.404480:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865216 00000100:00100000:2.0:1789760787.404504:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865216 00000100:00100000:2.0:1789760787.404740:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717466643752 time 717496123349 state 0x0 ref 1 00000100:00100000:2.0:1789760787.404760:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717496383765 time 717496123349 state 0x1 ref 1 00000100:00100000:2.0:1789760787.404809:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717496383765 time 717529415823 state 0x0 ref 1 00000100:00100000:2.0:1789760787.404825:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503865216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.404892:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79241216, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25ca8 00010000:00010000:2.0:1789760787.404903:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.404922:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.404928:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.404935:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.404950:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.411151:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986653 is committed 00002000:00010000:2.0:1789760787.412874:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79241216 -> 79245312 00002000:00010000:2.0:1789760787.412885:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154792 -> 154800 00010000:00010000:2.0:1789760787.412904:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.412918:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.412927:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.412935:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.412941:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.412952:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.412977:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.412985:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1907 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79241216->79245311] (req 79241216->79245311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.413118:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503865216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8289us (10589us total) trans 4294986653 rc 0/0 00000100:00100000:2.0:1789760787.413132:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19357 00000100:00100000:2.0:1789760787.413144:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717496383765 time 717529415823 state 0x2 ref 0 00000100:00100000:2.0:1789760787.431714:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.432423:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865344 00000100:00100000:1.0:1789760787.432438:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865344 00000100:00100000:1.0:1789760787.432518:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717496383765 time 717529415823 state 0x0 ref 1 00000100:00100000:1.0:1789760787.432553:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717529676239 time 717529415823 state 0x1 ref 1 00000100:00100000:1.0:1789760787.432590:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717529676239 time 717557196184 state 0x0 ref 1 00000100:00100000:1.0:1789760787.432604:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.432673:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79245312, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cb0 00010000:00010000:1.0:1789760787.432687:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.432709:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.432716:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.432724:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.432742:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.439014:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986654 is committed 00002000:00010000:1.0:1789760787.439168:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79245312 -> 79249408 00002000:00010000:1.0:1789760787.439174:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154800 -> 154808 00010000:00010000:1.0:1789760787.439192:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.439207:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.439216:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.439224:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.439230:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.439241:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.439277:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.439292:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f190e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79245312->79249407] (req 79245312->79249407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.439474:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6865us (7771us total) trans 4294986654 rc 0/0 00000100:00100000:1.0:1789760787.439487:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19358 00000100:00100000:1.0:1789760787.439500:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717529676239 time 717557196184 state 0x2 ref 0 00000100:00100000:1.0:1789760787.446445:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.446543:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865472 00000100:00100000:2.0:1789760787.446556:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865472 00000100:00100000:2.0:1789760787.446634:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717529676239 time 717557196184 state 0x0 ref 1 00000100:00100000:2.0:1789760787.446653:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717557456600 time 717557196184 state 0x1 ref 1 00000100:00100000:1.0:1789760787.446726:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717557456600 time 717571327196 state 0x0 ref 1 00000100:00100000:1.0:1789760787.446744:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503865472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.446812:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79249408, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cb8 00010000:00010000:1.0:1789760787.446824:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.446844:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.446849:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.446856:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.446873:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.458371:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986655 is committed 00002000:00010000:1.0:1789760787.458551:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79249408 -> 79253504 00002000:00010000:1.0:1789760787.458558:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154808 -> 154816 00010000:00010000:1.0:1789760787.458576:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.458592:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.458602:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.458610:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.458615:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.458626:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.458653:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.458662:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1915 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79249408->79253503] (req 79249408->79253503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.458793:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503865472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12045us (12354us total) trans 4294986655 rc 0/0 00000100:00100000:1.0:1789760787.458807:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19359 00000100:00100000:1.0:1789760787.458819:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717557456600 time 717571327196 state 0x2 ref 0 00000100:00100000:0.0:1789760787.463138:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.463239:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865600 00000100:00100000:2.0:1789760787.463255:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865600 00000100:00100000:2.0:1789760787.463329:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717557456600 time 717571327196 state 0x0 ref 1 00000100:00100000:2.0:1789760787.463350:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717571587612 time 717571327196 state 0x1 ref 1 00000100:00100000:2.0:1789760787.463386:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717571587612 time 717587991912 state 0x0 ref 1 00000100:00100000:2.0:1789760787.463436:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.463500:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79253504, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cc0 00010000:00010000:2.0:1789760787.463511:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.463541:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.463547:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.463556:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.463574:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.471338:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986656 is committed 00002000:00010000:3.0:1789760787.471503:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79253504 -> 79257600 00002000:00010000:3.0:1789760787.471509:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154816 -> 154824 00010000:00010000:3.0:1789760787.471536:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471552:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471877:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471888:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.471895:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471910:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471935:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471942:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f191c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79253504->79257599] (req 79253504->79257599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.472065:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8626us (8936us total) trans 4294986656 rc 0/0 00000100:00100000:3.0:1789760787.472076:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19360 00000100:00100000:3.0:1789760787.472088:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717571587612 time 717587991912 state 0x2 ref 0 00000100:00100000:3.0:1789760787.475952:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.476794:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865728 00000100:00100000:1.0:1789760787.476809:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865728 00000100:00100000:1.0:1789760787.476888:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717571587612 time 717587991912 state 0x0 ref 1 00000100:00100000:1.0:1789760787.476908:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717588252328 time 717587991912 state 0x1 ref 1 00000100:00100000:1.0:1789760787.476949:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717588252328 time 717601555351 state 0x0 ref 1 00000100:00100000:1.0:1789760787.476964:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503865728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.477181:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79257600, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cc8 00010000:00010000:1.0:1789760787.477193:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.477213:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.477217:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.477224:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.477241:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.489186:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986657 is committed 00002000:00010000:3.0:1789760787.489726:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79257600 -> 79261696 00002000:00010000:3.0:1789760787.489732:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154824 -> 154832 00010000:00010000:3.0:1789760787.489750:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.489764:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.489774:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.489786:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.489792:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.489802:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.489825:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.489832:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1923 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79257600->79261695] (req 79257600->79261695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.489960:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503865728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12990us (14016us total) trans 4294986657 rc 0/0 00000100:00100000:3.0:1789760787.489973:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19361 00000100:00100000:3.0:1789760787.489985:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717588252328 time 717601555351 state 0x2 ref 0 00000100:00100000:1.0:1789760787.498683:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.498833:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865856 00000100:00100000:0.0:1789760787.498848:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865856 00000100:00100000:0.0:1789760787.498926:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717588252328 time 717601555351 state 0x0 ref 1 00000100:00100000:0.0:1789760787.498947:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717601815767 time 717601555351 state 0x1 ref 1 00000100:00100000:0.0:1789760787.499015:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717601815767 time 717623620678 state 0x0 ref 1 00000100:00100000:0.0:1789760787.499107:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.499172:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79261696, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cd0 00010000:00010000:0.0:1789760787.499184:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.499205:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.499211:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.499219:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.499235:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.506928:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986658 is committed 00002000:00010000:0.0:1789760787.507086:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79261696 -> 79265792 00002000:00010000:0.0:1789760787.507093:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154832 -> 154840 00010000:00010000:0.0:1789760787.507112:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.507126:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.507136:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.507144:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.507149:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.507160:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.507187:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.507196:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f192a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79261696->79265791] (req 79261696->79265791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.507969:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503865856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8929us (9706us total) trans 4294986658 rc 0/0 00000100:00100000:0.0:1789760787.507985:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19362 00000100:00100000:0.0:1789760787.507996:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717601815767 time 717623620678 state 0x2 ref 0 00000100:00100000:2.0:1789760787.514686:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.514913:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503865984 00000100:00100000:1.0:1789760787.514926:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503865984 00000100:00100000:1.0:1789760787.515006:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717601815767 time 717623620678 state 0x0 ref 1 00000100:00100000:1.0:1789760787.515029:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717623881094 time 717623620678 state 0x1 ref 1 00000100:00100000:1.0:1789760787.515118:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717623881094 time 717639723684 state 0x0 ref 1 00000100:00100000:1.0:1789760787.515133:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503865984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.515198:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79265792, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cd8 00010000:00010000:1.0:1789760787.515209:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.515228:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.515233:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.515241:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.515271:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.526959:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986659 is committed 00002000:00010000:1.0:1789760787.527136:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79265792 -> 79269888 00002000:00010000:1.0:1789760787.527143:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154840 -> 154848 00010000:00010000:1.0:1789760787.527160:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.527176:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.527187:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.527197:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.527203:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.527214:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.527240:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.528997:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1931 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79265792->79269887] (req 79265792->79269887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.529489:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503865984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14350us (14809us total) trans 4294986659 rc 0/0 00000100:00100000:2.0:1789760787.529504:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19363 00000100:00100000:2.0:1789760787.529517:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717623881094 time 717639723684 state 0x2 ref 0 00000100:00100000:2.0:1789760787.535095:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.535693:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503866112 00000100:00100000:3.0:1789760787.535708:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503866112 00000100:00100000:3.0:1789760787.535789:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717623881094 time 717639723684 state 0x0 ref 1 00000100:00100000:3.0:1789760787.535808:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717639984100 time 717639723684 state 0x1 ref 1 00000100:00100000:3.0:1789760787.535880:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717639984100 time 717660485513 state 0x0 ref 1 00000100:00100000:3.0:1789760787.535896:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503866112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.535959:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79269888, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25ce0 00010000:00010000:3.0:1789760787.535971:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.535991:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.535997:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.536004:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.536022:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.547834:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986660 is committed 00002000:00010000:2.0:1789760787.551795:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79269888 -> 79273984 00002000:00010000:2.0:1789760787.551807:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154848 -> 154856 00010000:00010000:2.0:1789760787.551826:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.551841:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.551852:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.551861:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.551866:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.551882:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.551907:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.551916:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1938 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79269888->79273983] (req 79269888->79273983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.552111:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503866112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16209us (17023us total) trans 4294986660 rc 0/0 00000100:00100000:2.0:1789760787.552124:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19364 00000100:00100000:2.0:1789760787.552136:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717639984100 time 717660485513 state 0x2 ref 0 00000100:00100000:1.0:1789760787.559879:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.560202:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503866240 00000100:00100000:3.0:1789760787.560216:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503866240 00000100:00100000:3.0:1789760787.560345:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717639984100 time 717660485513 state 0x0 ref 1 00000100:00100000:3.0:1789760787.560364:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717660745929 time 717660485513 state 0x1 ref 1 00000100:00100000:3.0:1789760787.560494:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717660745929 time 717685099335 state 0x0 ref 1 00000100:00100000:3.0:1789760787.560508:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503866240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.560582:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79273984, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25ce8 00010000:00010000:3.0:1789760787.560594:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.560619:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.560624:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.560632:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.560650:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.573474:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986661 is committed 00002000:00010000:0.0:1789760787.574800:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79273984 -> 79278080 00002000:00010000:0.0:1789760787.574812:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154856 -> 154864 00010000:00010000:0.0:1789760787.574833:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574849:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574860:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574868:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.574874:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574885:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574911:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.574920:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f193f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79273984->79278079] (req 79273984->79278079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.575081:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503866240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14568us (15210us total) trans 4294986661 rc 0/0 00000100:00100000:0.0:1789760787.575095:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19365 00000100:00100000:0.0:1789760787.575107:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717660745929 time 717685099335 state 0x2 ref 0 00000100:00100000:2.0:1789760787.578497:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.578800:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503866368 00000100:00100000:0.0:1789760787.578813:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503866368 00000100:00100000:0.0:1789760787.578891:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717660745929 time 717685099335 state 0x0 ref 1 00000100:00100000:0.0:1789760787.578911:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717685359751 time 717685099335 state 0x1 ref 1 00000100:00100000:0.0:1789760787.578944:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717685359751 time 717703550370 state 0x0 ref 1 00000100:00100000:0.0:1789760787.578958:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503866368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.579029:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79278080, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cf0 00010000:00010000:0.0:1789760787.579041:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.579060:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.579066:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.579074:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.579092:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.589113:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986662 is committed 00002000:00010000:0.0:1789760787.589583:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79278080 -> 79282176 00002000:00010000:0.0:1789760787.589590:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154864 -> 154872 00010000:00010000:0.0:1789760787.589609:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.589922:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.589931:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.589940:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.589945:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.589955:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.589981:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.589989:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1946 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79278080->79282175] (req 79278080->79282175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.590600:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503866368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11633us (12108us total) trans 4294986662 rc 0/0 00000100:00100000:0.0:1789760787.590623:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19366 00000100:00100000:0.0:1789760787.590635:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717685359751 time 717703550370 state 0x2 ref 0 00000100:00100000:2.0:1789760787.596635:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.597094:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503866496 00000100:00100000:0.0:1789760787.597108:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503866496 00000100:00100000:0.0:1789760787.597186:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717685359751 time 717703550370 state 0x0 ref 1 00000100:00100000:0.0:1789760787.597213:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717703810786 time 717703550370 state 0x1 ref 1 00000100:00100000:0.0:1789760787.597301:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717703810786 time 717721906581 state 0x0 ref 1 00000100:00100000:0.0:1789760787.597317:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503866496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.597385:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79282176, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25cf8 00010000:00010000:0.0:1789760787.597441:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.597463:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.597469:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.597477:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.597494:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.604979:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986663 is committed 00002000:00010000:0.0:1789760787.605118:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79282176 -> 79286272 00002000:00010000:0.0:1789760787.605124:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154872 -> 154880 00010000:00010000:0.0:1789760787.605138:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.605156:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.605165:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.605172:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.605177:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.605186:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.605206:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.605214:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f194d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79282176->79286271] (req 79282176->79286271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.605317:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503866496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7998us (8693us total) trans 4294986663 rc 0/0 00000100:00100000:0.0:1789760787.605327:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19367 00000100:00100000:0.0:1789760787.605337:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717703810786 time 717721906581 state 0x2 ref 0 00000100:00100000:1.0:1789760787.615959:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.616092:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503866624 00000100:00100000:2.0:1789760787.616104:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503866624 00000100:00100000:2.0:1789760787.616181:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717703810786 time 717721906581 state 0x0 ref 1 00000100:00100000:2.0:1789760787.616214:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717722166997 time 717721906581 state 0x1 ref 1 00000100:00100000:2.0:1789760787.616248:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717722166997 time 717740853691 state 0x0 ref 1 00000100:00100000:2.0:1789760787.616262:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503866624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.616331:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79286272, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d00 00010000:00010000:2.0:1789760787.616344:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.616364:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.616370:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.616378:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.616447:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.624768:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986664 is committed 00002000:00010000:0.0:1789760787.624911:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79286272 -> 79290368 00002000:00010000:0.0:1789760787.624916:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154880 -> 154888 00010000:00010000:0.0:1789760787.624929:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.624943:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.624951:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.624958:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.624964:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.624973:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.624993:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.625001:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f1954 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79286272->79290367] (req 79286272->79290367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.625096:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503866624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8832us (9149us total) trans 4294986664 rc 0/0 00000100:00100000:0.0:1789760787.625106:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19368 00000100:00100000:0.0:1789760787.625116:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717722166997 time 717740853691 state 0x2 ref 0 00000100:00100000:2.0:1789760787.633145:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.637356:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503866752 00000100:00100000:3.0:1789760787.637371:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503866752 00000100:00100000:3.0:1789760787.637457:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717722166997 time 717740853691 state 0x0 ref 1 00000100:00100000:3.0:1789760787.637473:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717741114107 time 717740853691 state 0x1 ref 1 00000100:00100000:3.0:1789760787.637497:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717741114107 time 717762103745 state 0x0 ref 1 00000100:00100000:3.0:1789760787.637510:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503866752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.637565:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79290368, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d08 00010000:00010000:3.0:1789760787.637575:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.637593:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.637597:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.637605:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.637619:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.645314:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986665 is committed 00002000:00010000:3.0:1789760787.646170:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79290368 -> 79294464 00002000:00010000:3.0:1789760787.646177:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154888 -> 154896 00010000:00010000:3.0:1789760787.646190:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.646203:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.646211:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.646219:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.646224:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.646233:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.646264:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.646277:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f195b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79290368->79294463] (req 79290368->79294463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.646457:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503866752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8884us (13257us total) trans 4294986665 rc 0/0 00000100:00100000:3.0:1789760787.646468:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19369 00000100:00100000:3.0:1789760787.646478:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717741114107 time 717762103745 state 0x2 ref 0 00000100:00100000:2.0:1789760787.652812:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.652902:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503866880 00000100:00100000:0.0:1789760787.652912:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503866880 00000100:00100000:0.0:1789760787.652972:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717741114107 time 717762103745 state 0x0 ref 1 00000100:00100000:0.0:1789760787.652988:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717762364161 time 717762103745 state 0x1 ref 1 00000100:00100000:0.0:1789760787.653022:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717762364161 time 717777628562 state 0x0 ref 1 00000100:00100000:0.0:1789760787.653034:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503866880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.653082:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79294464, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d10 00010000:00010000:0.0:1789760787.653090:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.653105:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.653109:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.653115:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.653128:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.662953:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986666 is committed 00002000:00010000:1.0:1789760787.664231:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79294464 -> 79298560 00002000:00010000:1.0:1789760787.664239:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154896 -> 154904 00010000:00010000:1.0:1789760787.664852:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.664867:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.664876:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.664890:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.664895:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.664904:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.664926:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.664933:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1962 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79294464->79298559] (req 79294464->79298559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.665268:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503866880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12230us (12463us total) trans 4294986666 rc 0/0 00000100:00100000:1.0:1789760787.665279:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19370 00000100:00100000:1.0:1789760787.665290:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717762364161 time 717777628562 state 0x2 ref 0 00000100:00100000:1.0:1789760787.675974:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.676832:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867008 00000100:00100000:0.0:1789760787.676845:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867008 00000100:00100000:0.0:1789760787.676906:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717762364161 time 717777628562 state 0x0 ref 1 00000100:00100000:0.0:1789760787.676922:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717777888978 time 717777628562 state 0x1 ref 1 00000100:00100000:0.0:1789760787.676962:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717777888978 time 717801568983 state 0x0 ref 1 00000100:00100000:0.0:1789760787.676974:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.677021:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79298560, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d18 00010000:00010000:0.0:1789760787.677030:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.677045:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.677049:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.677056:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.677068:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.687431:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986667 is committed 00002000:00010000:1.0:1789760787.688267:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79298560 -> 79302656 00002000:00010000:1.0:1789760787.688275:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154904 -> 154912 00010000:00010000:1.0:1789760787.688289:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.688301:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.688313:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.688319:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.688323:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.688332:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.688352:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.688359:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1969 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79298560->79302655] (req 79298560->79302655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.688520:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11542us (12550us total) trans 4294986667 rc 0/0 00000100:00100000:1.0:1789760787.688540:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19371 00000100:00100000:1.0:1789760787.688550:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717777888978 time 717801568983 state 0x2 ref 0 00000100:00100000:2.0:1789760787.694950:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.695114:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867136 00000100:00100000:0.0:1789760787.695124:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867136 00000100:00100000:0.0:1789760787.695189:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717777888978 time 717801568983 state 0x0 ref 1 00000100:00100000:0.0:1789760787.695204:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717801829399 time 717801568983 state 0x1 ref 1 00000100:00100000:0.0:1789760787.695234:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717801829399 time 717819839921 state 0x0 ref 1 00000100:00100000:0.0:1789760787.695245:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.695299:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79302656, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d20 00010000:00010000:0.0:1789760787.695308:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.695323:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.695328:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.695333:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.695347:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.704851:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986668 is committed 00002000:00010000:0.0:1789760787.705144:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79302656 -> 79306752 00002000:00010000:0.0:1789760787.705151:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154912 -> 154920 00010000:00010000:0.0:1789760787.705181:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.705194:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.705202:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.705209:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.705213:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.705222:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.705241:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.705738:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1970 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79302656->79306751] (req 79302656->79306751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.705833:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10585us (10888us total) trans 4294986668 rc 0/0 00000100:00100000:0.0:1789760787.705844:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19372 00000100:00100000:0.0:1789760787.705853:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717801829399 time 717819839921 state 0x2 ref 0 00000100:00100000:2.0:1789760787.710554:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.710717:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867264 00000100:00100000:3.0:1789760787.710728:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867264 00000100:00100000:3.0:1789760787.710779:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717801829399 time 717819839921 state 0x0 ref 1 00000100:00100000:3.0:1789760787.710795:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717820100337 time 717819839921 state 0x1 ref 1 00000100:00100000:3.0:1789760787.710816:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717820100337 time 717835423498 state 0x0 ref 1 00000100:00100000:3.0:1789760787.710827:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503867264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.710871:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79306752, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d28 00010000:00010000:3.0:1789760787.710880:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.710894:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.710899:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.710905:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.710919:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.716141:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986669 is committed 00002000:00010000:3.0:1789760787.716362:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79306752 -> 79310848 00002000:00010000:3.0:1789760787.716368:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154920 -> 154928 00010000:00010000:3.0:1789760787.716381:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.716413:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.716421:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.716428:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.716432:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.716441:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.716462:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.716474:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1977 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79306752->79310847] (req 79306752->79310847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.716585:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503867264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5754us (6038us total) trans 4294986669 rc 0/0 00000100:00100000:3.0:1789760787.716595:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19373 00000100:00100000:3.0:1789760787.716604:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717820100337 time 717835423498 state 0x2 ref 0 00000100:00100000:1.0:1789760787.721089:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.721207:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867392 00000100:00100000:2.0:1789760787.721217:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867392 00000100:00100000:2.0:1789760787.721279:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717820100337 time 717835423498 state 0x0 ref 1 00000100:00100000:2.0:1789760787.721299:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717835683914 time 717835423498 state 0x1 ref 1 00000100:00100000:2.0:1789760787.721336:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717835683914 time 717845942378 state 0x0 ref 1 00000100:00100000:2.0:1789760787.721349:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.721629:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79310848, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d30 00010000:00010000:2.0:1789760787.721639:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.721654:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.721659:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.721665:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.721680:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.728809:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986670 is committed 00002000:00010000:2.0:1789760787.729037:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79310848 -> 79314944 00002000:00010000:2.0:1789760787.729043:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154928 -> 154936 00010000:00010000:2.0:1789760787.729056:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.729069:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.729078:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.729085:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.729090:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.729104:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.729124:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.729131:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f197e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79310848->79314943] (req 79310848->79314943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.729303:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7948us (8217us total) trans 4294986670 rc 0/0 00000100:00100000:2.0:1789760787.729315:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19374 00000100:00100000:2.0:1789760787.729326:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717835683914 time 717845942378 state 0x2 ref 0 00000100:00100000:2.0:1789760787.740033:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.740122:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867520 00000100:00100000:1.0:1789760787.740132:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867520 00000100:00100000:1.0:1789760787.740637:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717835683914 time 717845942378 state 0x0 ref 1 00000100:00100000:1.0:1789760787.740659:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717846202794 time 717845942378 state 0x1 ref 1 00000100:00100000:1.0:1789760787.740691:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717846202794 time 717865297821 state 0x0 ref 1 00000100:00100000:1.0:1789760787.740706:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.740752:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79314944, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d38 00010000:00010000:1.0:1789760787.740761:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.740776:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.740781:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.740787:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.740799:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.749744:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986671 is committed 00002000:00010000:2.0:1789760787.749874:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79314944 -> 79319040 00002000:00010000:2.0:1789760787.749879:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154936 -> 154944 00010000:00010000:2.0:1789760787.749893:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.749905:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.749915:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.749927:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.749932:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.749941:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.749960:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.749969:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1985 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79314944->79319039] (req 79314944->79319039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.750059:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9352us (10032us total) trans 4294986671 rc 0/0 00000100:00100000:2.0:1789760787.750070:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19375 00000100:00100000:2.0:1789760787.750079:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717846202794 time 717865297821 state 0x2 ref 0 00000100:00100000:2.0:1789760787.754801:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.754913:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867648 00000100:00100000:0.0:1789760787.754924:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867648 00000100:00100000:0.0:1789760787.754987:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717846202794 time 717865297821 state 0x0 ref 1 00000100:00100000:0.0:1789760787.755005:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717865558237 time 717865297821 state 0x1 ref 1 00000100:00100000:0.0:1789760787.755037:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717865558237 time 717879643327 state 0x0 ref 1 00000100:00100000:0.0:1789760787.755049:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.755331:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79319040, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d40 00010000:00010000:0.0:1789760787.755341:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.755363:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.755368:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.755374:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.755387:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.760789:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986672 is committed 00002000:00010000:0.0:1789760787.761016:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79319040 -> 79323136 00002000:00010000:0.0:1789760787.761020:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154944 -> 154952 00010000:00010000:0.0:1789760787.761031:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.761039:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.761044:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.761049:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.761052:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.761058:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.761075:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.761080:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f198c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79319040->79323135] (req 79319040->79323135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.761163:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6111us (6367us total) trans 4294986672 rc 0/0 00000100:00100000:0.0:1789760787.761177:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19376 00000100:00100000:0.0:1789760787.761188:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717865558237 time 717879643327 state 0x2 ref 0 00000100:00100000:1.0:1789760787.766348:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.766572:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867776 00000100:00100000:3.0:1789760787.766583:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867776 00000100:00100000:3.0:1789760787.766636:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717865558237 time 717879643327 state 0x0 ref 1 00000100:00100000:3.0:1789760787.766651:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717879903743 time 717879643327 state 0x1 ref 1 00000100:00100000:3.0:1789760787.766685:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717879903743 time 717891291181 state 0x0 ref 1 00000100:00100000:3.0:1789760787.766698:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.766745:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79323136, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d48 00010000:00010000:3.0:1789760787.766754:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.766770:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.766775:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.766781:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.766801:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.772908:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986673 is committed 00002000:00010000:3.0:1789760787.773043:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79323136 -> 79327232 00002000:00010000:3.0:1789760787.773050:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154952 -> 154960 00010000:00010000:3.0:1789760787.773066:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.773082:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.773095:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.773103:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.773108:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.773117:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.773139:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.773146:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1993 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79323136->79327231] (req 79323136->79327231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.773260:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503867776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6559us (6917us total) trans 4294986673 rc 0/0 00000100:00100000:3.0:1789760787.773271:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19377 00000100:00100000:3.0:1789760787.773284:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717879903743 time 717891291181 state 0x2 ref 0 00000100:00100000:0.0:1789760787.775610:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.775743:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503867904 00000100:00100000:1.0:1789760787.775752:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503867904 00000100:00100000:1.0:1789760787.775815:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717879903743 time 717891291181 state 0x0 ref 1 00000100:00100000:1.0:1789760787.775834:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717891551597 time 717891291181 state 0x1 ref 1 00000100:00100000:1.0:1789760787.775863:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717891551597 time 717900469757 state 0x0 ref 1 00000100:00100000:1.0:1789760787.775875:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503867904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.775930:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79327232, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x25d50 00010000:00010000:1.0:1789760787.775939:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.775959:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.775963:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.775970:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.775987:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.780955:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986674 is committed 00002000:00010000:1.0:1789760787.781865:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79327232 -> 79331328 00002000:00010000:1.0:1789760787.781872:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:1.0:1789760787.781874:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00002000:00010000:1.0:1789760787.781876:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154960 -> 154968 00010000:00010000:1.0:1789760787.781890:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.781902:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.781912:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.781921:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.781926:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.781935:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.781955:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.781964:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f199a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79327232->79331327] (req 79327232->79331327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.782183:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503867904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6303us (6580us total) trans 4294986674 rc 0/0 00000100:00100000:1.0:1789760787.782194:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19378 00000100:00100000:1.0:1789760787.782204:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717891551597 time 717900469757 state 0x2 ref 0 00000100:00100000:2.0:1789760787.786091:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.786187:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868032 00000100:00100000:1.0:1789760787.786197:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868032 00000100:00100000:1.0:1789760787.786309:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717891551597 time 717900469757 state 0x0 ref 1 00000100:00100000:1.0:1789760787.786327:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717900730173 time 717900469757 state 0x1 ref 1 00000100:00100000:1.0:1789760787.786644:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717900730173 time 717910995607 state 0x0 ref 1 00000100:00100000:1.0:1789760787.786664:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503868032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.786714:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79331328, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d58 00010000:00010000:1.0:1789760787.786723:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.786741:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.786745:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.786751:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.786765:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.792359:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986675 is committed 00002000:00010000:1.0:1789760787.792490:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79331328 -> 79335424 00002000:00010000:1.0:1789760787.792495:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154968 -> 154976 00010000:00010000:1.0:1789760787.792507:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.792534:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.792544:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.792551:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.792557:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.792725:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.792745:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.792753:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79331328->79335423] (req 79331328->79335423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.792840:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503868032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6175us (6754us total) trans 4294986675 rc 0/0 00000100:00100000:1.0:1789760787.792851:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19379 00000100:00100000:1.0:1789760787.792860:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717900730173 time 717910995607 state 0x2 ref 0 00000100:00100000:2.0:1789760787.796351:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.796469:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868160 00000100:00100000:0.0:1789760787.796478:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868160 00000100:00100000:0.0:1789760787.796548:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717900730173 time 717910995607 state 0x0 ref 1 00000100:00100000:0.0:1789760787.796576:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717911256023 time 717910995607 state 0x1 ref 1 00000100:00100000:0.0:1789760787.796604:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717911256023 time 717921210613 state 0x0 ref 1 00000100:00100000:0.0:1789760787.796617:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503868160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.796664:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79335424, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d60 00010000:00010000:0.0:1789760787.796673:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.796695:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760787.796700:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.796706:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.796720:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.803443:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986676 is committed 00002000:00010000:0.0:1789760787.803667:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79335424 -> 79339520 00002000:00010000:0.0:1789760787.803672:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154976 -> 154984 00010000:00010000:0.0:1789760787.803684:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.803696:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.803704:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.803711:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.803716:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.803725:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.803751:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.803759:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f19a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79335424->79339519] (req 79335424->79339519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.803888:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503868160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7269us (7542us total) trans 4294986676 rc 0/0 00000100:00100000:0.0:1789760787.803899:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19380 00000100:00100000:0.0:1789760787.803909:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717911256023 time 717921210613 state 0x2 ref 0 00000100:00100000:0.0:1789760787.811469:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.811642:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868288 00000100:00100000:3.0:1789760787.811652:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868288 00000100:00100000:3.0:1789760787.811702:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717911256023 time 717921210613 state 0x0 ref 1 00000100:00100000:3.0:1789760787.811718:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717921471029 time 717921210613 state 0x1 ref 1 00000100:00100000:3.0:1789760787.811748:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717921471029 time 717936355593 state 0x0 ref 1 00000100:00100000:3.0:1789760787.811760:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503868288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.811801:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79339520, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d68 00010000:00010000:3.0:1789760787.811810:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.811826:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.811830:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.811836:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.811849:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.821959:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986677 is committed 00002000:00010000:3.0:1789760787.822130:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79339520 -> 79343616 00002000:00010000:3.0:1789760787.822135:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154984 -> 154992 00010000:00010000:3.0:1789760787.822149:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.822167:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.822175:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.822183:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.822188:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.822197:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.822217:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.822224:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79339520->79343615] (req 79339520->79343615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.822331:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503868288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10567us (10872us total) trans 4294986677 rc 0/0 00000100:00100000:3.0:1789760787.822343:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19381 00000100:00100000:3.0:1789760787.822352:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717921471029 time 717936355593 state 0x2 ref 0 00000100:00100000:2.0:1789760787.827333:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.827415:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868416 00000100:00100000:1.0:1789760787.827427:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868416 00000100:00100000:1.0:1789760787.827492:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717921471029 time 717936355593 state 0x0 ref 1 00000100:00100000:1.0:1789760787.827508:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717936616009 time 717936355593 state 0x1 ref 1 00000100:00100000:1.0:1789760787.827546:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717936616009 time 717952153005 state 0x0 ref 1 00000100:00100000:1.0:1789760787.827559:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503868416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.827607:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79343616, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d70 00010000:00010000:1.0:1789760787.827616:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.827631:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.827636:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.827643:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.827656:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.833284:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986678 is committed 00002000:00010000:1.0:1789760787.833479:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79343616 -> 79347712 00002000:00010000:1.0:1789760787.833484:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 154992 -> 155000 00010000:00010000:1.0:1789760787.833498:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.833510:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.833519:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.833536:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.833541:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.833555:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.833575:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.833582:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79343616->79347711] (req 79343616->79347711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.833714:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503868416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6152us (6384us total) trans 4294986678 rc 0/0 00000100:00100000:1.0:1789760787.833725:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19382 00000100:00100000:1.0:1789760787.833734:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717936616009 time 717952153005 state 0x2 ref 0 00000100:00100000:2.0:1789760787.845849:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.845989:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868544 00000100:00100000:1.0:1789760787.845999:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868544 00000100:00100000:1.0:1789760787.846057:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717936616009 time 717952153005 state 0x0 ref 1 00000100:00100000:1.0:1789760787.846072:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717952413421 time 717952153005 state 0x1 ref 1 00000100:00100000:1.0:1789760787.846104:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717952413421 time 717970710427 state 0x0 ref 1 00000100:00100000:1.0:1789760787.846117:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503868544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.846165:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79347712, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d78 00010000:00010000:1.0:1789760787.846183:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.846200:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760787.846205:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.846211:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.846225:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.852768:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986679 is committed 00002000:00010000:2.0:1789760787.852957:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79347712 -> 79351808 00002000:00010000:2.0:1789760787.852962:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155000 -> 155008 00010000:00010000:2.0:1789760787.852975:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.852986:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.852993:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.853000:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.853005:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.853012:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.853031:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.853039:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f19bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79347712->79351807] (req 79347712->79351807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.853138:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503868544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7019us (7293us total) trans 4294986679 rc 0/0 00000100:00100000:2.0:1789760787.853151:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19383 00000100:00100000:2.0:1789760787.853161:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717952413421 time 717970710427 state 0x2 ref 0 00000100:00100000:0.0:1789760787.861840:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.862041:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868672 00000100:00100000:2.0:1789760787.862052:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868672 00000100:00100000:2.0:1789760787.862109:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717952413421 time 717970710427 state 0x0 ref 1 00000100:00100000:2.0:1789760787.862122:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717970970843 time 717970710427 state 0x1 ref 1 00000100:00100000:2.0:1789760787.862155:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717970970843 time 717986761476 state 0x0 ref 1 00000100:00100000:2.0:1789760787.862169:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503868672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.862214:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79351808, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d80 00010000:00010000:2.0:1789760787.862223:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.862239:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760787.862243:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.862253:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.862269:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.867789:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986680 is committed 00002000:00010000:2.0:1789760787.867948:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79351808 -> 79355904 00002000:00010000:2.0:1789760787.867953:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155008 -> 155016 00010000:00010000:2.0:1789760787.867967:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.867980:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.867989:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.867996:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.868000:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.868009:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.868029:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.868036:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79351808->79355903] (req 79351808->79355903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.868134:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503868672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5964us (6300us total) trans 4294986680 rc 0/0 00000100:00100000:2.0:1789760787.868145:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19384 00000100:00100000:2.0:1789760787.868154:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717970970843 time 717986761476 state 0x2 ref 0 00000100:00100000:0.0:1789760787.875092:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.875735:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868800 00000100:00100000:3.0:1789760787.875746:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868800 00000100:00100000:3.0:1789760787.875801:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717970970843 time 717986761476 state 0x0 ref 1 00000100:00100000:3.0:1789760787.875815:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717987021892 time 717986761476 state 0x1 ref 1 00000100:00100000:3.0:1789760787.875921:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717987021892 time 718000526022 state 0x0 ref 1 00000100:00100000:3.0:1789760787.875934:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503868800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760787.875982:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79355904, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d88 00010000:00010000:3.0:1789760787.875992:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.876009:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760787.876013:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.876019:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.876033:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.883894:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986681 is committed 00002000:00010000:3.0:1789760787.884079:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79355904 -> 79360000 00002000:00010000:3.0:1789760787.884085:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155016 -> 155024 00010000:00010000:3.0:1789760787.884098:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.884110:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.884118:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.884126:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760787.884131:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.884140:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.884647:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.884659:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f19cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79355904->79359999] (req 79355904->79359999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.884923:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab6ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503868800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8985us (9832us total) trans 4294986681 rc 0/0 00000100:00100000:3.0:1789760787.884934:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19385 00000100:00100000:3.0:1789760787.884944:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717987021892 time 718000526022 state 0x2 ref 0 00000100:00100000:1.0:1789760787.895805:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.896361:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503868928 00000100:00100000:2.0:1789760787.896372:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503868928 00000100:00100000:2.0:1789760787.896645:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 717987021892 time 718000526022 state 0x0 ref 1 00000100:00100000:2.0:1789760787.896661:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718000786438 time 718000526022 state 0x1 ref 1 00000100:00100000:2.0:1789760787.896729:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718000786438 time 718021335193 state 0x0 ref 1 00000100:00100000:2.0:1789760787.896742:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503868928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.896797:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79360000, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d90 00010000:00010000:2.0:1789760787.896806:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896820:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760787.896824:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.896830:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896843:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.902295:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986682 is committed 00002000:00010000:2.0:1789760787.902656:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79360000 -> 79364096 00002000:00010000:2.0:1789760787.902662:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155024 -> 155032 00010000:00010000:2.0:1789760787.902674:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.902693:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.902702:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.902710:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.902715:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.902724:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.902745:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.902753:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79360000->79364095] (req 79360000->79364095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.902849:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503868928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6104us (7047us total) trans 4294986682 rc 0/0 00000100:00100000:2.0:1789760787.902859:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19386 00000100:00100000:2.0:1789760787.902869:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718000786438 time 718021335193 state 0x2 ref 0 00000100:00100000:1.0:1789760787.908790:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.909161:0:6505:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869056 00000100:00100000:2.0:1789760787.909172:0:6505:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869056 00000100:00100000:2.0:1789760787.909231:0:6505:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 492 00000100:00100000:2.0:1789760787.909241:0:6505:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503869056:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760787.909470:0:6505:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503869056:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 230us (688us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.909479:0:6505:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 492 00000100:00100000:0.0:1789760787.912730:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.916113:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869184 00000100:00100000:3.0:1789760787.916127:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869184 00000100:00100000:3.0:1789760787.916205:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 981 00000100:00100000:3.0:1789760787.916214:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503869184:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760787.916295:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5deab69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503869184:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 82us (3573us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.916303:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 981 00000100:00080000:0.0:1789760787.916444:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760787.916487:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.916513:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.916586:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869568 00000100:00100000:3.0:1789760787.916590:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869568 00000100:00100000:3.0:1789760787.916625:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 982 00000100:00100000:3.0:1789760787.916633:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503869568:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760787.917359:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869312 00000100:00100000:2.0:1789760787.917365:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869312 00000100:00100000:2.0:1789760787.917423:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718000786438 time 718021335193 state 0x0 ref 1 00000100:00100000:2.0:1789760787.917436:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718021595609 time 718021335193 state 0x1 ref 1 00000100:00100000:2.0:1789760787.917458:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718021595609 time 718042065352 state 0x0 ref 1 00000100:00100000:2.0:1789760787.917473:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503869312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.917514:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79364096, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25d98 00010000:00010000:2.0:1789760787.917540:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.917556:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760787.917561:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.917567:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.917580:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760787.921560:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760787.922991:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503869568:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 6348us (6493us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.923005:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 982 00000100:00100000:1.0:1789760787.923068:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.923148:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760787.926288:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869440 00000100:00100000:3.0:1789760787.926304:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869440 00000100:00100000:3.0:1789760787.926464:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869696 00000100:00100000:3.0:1789760787.926469:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869696 00000100:00100000:1.0:1789760787.926546:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 983 00000100:00100000:1.0:1789760787.926558:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503869440:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:1.0:1789760787.926651:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503869440:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 95us (3585us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.926659:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 983 00000100:00080000:0.0:1789760787.927082:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760787.927282:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 984 00000100:00100000:3.0:1789760787.927290:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503869696:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760787.927331:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503869696:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 43us (4184us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.927338:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 984 00000001:00080000:1.0:1789760787.935452:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986683 is committed 00002000:00010000:0.0:1789760787.935731:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79364096 -> 79368192 00002000:00010000:0.0:1789760787.935736:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155032 -> 155040 00010000:00010000:0.0:1789760787.935806:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.935821:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.935831:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.935839:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.935844:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.935853:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.935873:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.935882:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f19d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79364096->79368191] (req 79364096->79368191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.935978:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503869312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18506us (19459us total) trans 4294986683 rc 0/0 00000100:00100000:0.0:1789760787.935988:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19387 00000100:00100000:0.0:1789760787.935998:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718021595609 time 718042065352 state 0x2 ref 0 00000100:00100000:1.0:1789760787.938936:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.940535:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869824 00000100:00100000:0.0:1789760787.940548:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869824 00000100:00100000:0.0:1789760787.940629:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718021595609 time 718042065352 state 0x0 ref 1 00000100:00100000:0.0:1789760787.940644:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718042325768 time 718042065352 state 0x1 ref 1 00000100:00100000:0.0:1789760787.940675:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718042325768 time 718065281556 state 0x0 ref 1 00000100:00100000:0.0:1789760787.940688:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503869824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.940740:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79368192, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25da0 00010000:00010000:0.0:1789760787.940756:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.940773:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760787.940777:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.940784:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.940798:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.945747:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986684 is committed 00002000:00010000:1.0:1789760787.945928:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79368192 -> 79372288 00002000:00010000:1.0:1789760787.945933:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155040 -> 155048 00010000:00010000:1.0:1789760787.945946:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.946025:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.946035:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.946043:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.946047:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.946057:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.946077:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.946084:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f19e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79368192->79372287] (req 79368192->79372287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.946212:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503869824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5521us (7280us total) trans 4294986684 rc 0/0 00000100:00100000:1.0:1789760787.946222:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19388 00000100:00100000:1.0:1789760787.946232:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718042325768 time 718065281556 state 0x2 ref 0 00000100:00100000:3.0:1789760787.949732:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.949919:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503869952 00000100:00100000:0.0:1789760787.949930:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503869952 00000100:00100000:0.0:1789760787.949988:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718042325768 time 718065281556 state 0x0 ref 1 00000100:00100000:0.0:1789760787.950004:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718065541972 time 718065281556 state 0x1 ref 1 00000100:00100000:1.0:1789760787.950077:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718065541972 time 718074680240 state 0x0 ref 1 00000100:00100000:1.0:1789760787.950093:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc12850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503869952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.950141:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79372288, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25da8 00010000:00010000:1.0:1789760787.950150:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950165:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760787.950169:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.950175:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950188:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.954665:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986685 is committed 00002000:00010000:1.0:1789760787.955171:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79372288 -> 79376384 00002000:00010000:1.0:1789760787.955176:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155048 -> 155056 00010000:00010000:1.0:1789760787.955189:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.955201:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.955211:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.955224:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.955228:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.955238:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.955368:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.955378:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79372288->79376383] (req 79372288->79376383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.955705:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc12850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503869952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5610us (5976us total) trans 4294986685 rc 0/0 00000100:00100000:1.0:1789760787.955716:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19389 00000100:00100000:1.0:1789760787.955726:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718065541972 time 718074680240 state 0x2 ref 0 00000100:00100000:3.0:1789760787.959808:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760787.959903:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870080 00000100:00100000:2.0:1789760787.959912:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870080 00000100:00100000:2.0:1789760787.959968:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718065541972 time 718074680240 state 0x0 ref 1 00000100:00100000:2.0:1789760787.959984:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718074940656 time 718074680240 state 0x1 ref 1 00000100:00100000:2.0:1789760787.960679:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718074940656 time 718085281682 state 0x0 ref 1 00000100:00100000:2.0:1789760787.960696:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760787.960744:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79376384, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25db0 00010000:00010000:2.0:1789760787.960753:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.960768:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760787.960773:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.960780:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.960794:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.966123:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986686 is committed 00002000:00010000:2.0:1789760787.966375:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79376384 -> 79380480 00002000:00010000:2.0:1789760787.966381:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155056 -> 155064 00010000:00010000:2.0:1789760787.966763:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.966782:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.966791:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.966854:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760787.966859:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.966869:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.966888:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.966895:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f19ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79376384->79380479] (req 79376384->79380479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.966988:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6291us (7185us total) trans 4294986686 rc 0/0 00000100:00100000:2.0:1789760787.966998:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19390 00000100:00100000:2.0:1789760787.967008:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718074940656 time 718085281682 state 0x2 ref 0 00000100:00100000:1.0:1789760787.971140:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.971718:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870208 00000100:00100000:0.0:1789760787.971728:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870208 00000100:00100000:0.0:1789760787.971788:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718074940656 time 718085281682 state 0x0 ref 1 00000100:00100000:0.0:1789760787.971800:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718085542098 time 718085281682 state 0x1 ref 1 00000100:00100000:0.0:1789760787.971835:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718085542098 time 718096441630 state 0x0 ref 1 00000100:00100000:0.0:1789760787.971848:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.971894:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79380480, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25db8 00010000:00010000:0.0:1789760787.971903:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.971923:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760787.971929:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.971936:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.971948:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.977569:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986687 is committed 00002000:00010000:0.0:1789760787.978515:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79380480 -> 79384576 00002000:00010000:0.0:1789760787.978529:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155064 -> 155072 00010000:00010000:0.0:1789760787.978544:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.978556:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.978565:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.978577:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760787.978583:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.978591:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.978612:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.978620:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f19f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79380480->79384575] (req 79380480->79384575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.978722:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6871us (7587us total) trans 4294986687 rc 0/0 00000100:00100000:0.0:1789760787.978731:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19391 00000100:00100000:0.0:1789760787.978741:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718085542098 time 718096441630 state 0x2 ref 0 00010000:00010000:2.0:1789760787.986914:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760787.988840:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760787.988932:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870336 00000100:00100000:1.0:1789760787.988942:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870336 00000100:00100000:1.0:1789760787.988997:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718085542098 time 718096441630 state 0x0 ref 1 00000100:00100000:1.0:1789760787.989011:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718096702046 time 718096441630 state 0x1 ref 1 00000100:00100000:1.0:1789760787.989043:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718096702046 time 718113649446 state 0x0 ref 1 00000100:00100000:1.0:1789760787.989056:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760787.989101:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79384576, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25dc0 00010000:00010000:1.0:1789760787.989109:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.989123:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760787.989128:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.989134:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.989146:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.993735:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986688 is committed 00002000:00010000:1.0:1789760787.993942:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79384576 -> 79388672 00002000:00010000:1.0:1789760787.993947:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155072 -> 155080 00010000:00010000:1.0:1789760787.993960:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.993973:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.993982:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.993990:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760787.993995:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.994004:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.994023:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.994032:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f19fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79384576->79388671] (req 79384576->79388671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.994171:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5112us (5335us total) trans 4294986688 rc 0/0 00000100:00100000:1.0:1789760787.994182:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19392 00000100:00100000:1.0:1789760787.994191:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718096702046 time 718113649446 state 0x2 ref 0 00000100:00100000:1.0:1789760787.998856:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760787.998953:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870464 00000100:00100000:0.0:1789760787.998963:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870464 00000100:00100000:0.0:1789760787.999016:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718096702046 time 718113649446 state 0x0 ref 1 00000100:00100000:0.0:1789760787.999030:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718113909862 time 718113649446 state 0x1 ref 1 00000100:00100000:0.0:1789760787.999053:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718113909862 time 718123660838 state 0x0 ref 1 00000100:00100000:0.0:1789760787.999065:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760787.999114:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79388672, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25dc8 00010000:00010000:0.0:1789760787.999122:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.999136:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760787.999140:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.999146:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.999158:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.005204:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986689 is committed 00002000:00010000:0.0:1789760788.005312:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79388672 -> 79392768 00002000:00010000:0.0:1789760788.005318:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155080 -> 155088 00010000:00010000:0.0:1789760788.005331:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.005343:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.005352:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.005358:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.005362:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.005371:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.005643:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.005652:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79388672->79392767] (req 79388672->79392767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.005751:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6682us (6899us total) trans 4294986689 rc 0/0 00000100:00100000:0.0:1789760788.005763:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19393 00000100:00100000:0.0:1789760788.005772:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718113909862 time 718123660838 state 0x2 ref 0 00000100:00100000:1.0:1789760788.008864:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.009013:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870592 00000100:00100000:3.0:1789760788.009023:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870592 00000100:00100000:3.0:1789760788.009097:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718113909862 time 718123660838 state 0x0 ref 1 00000100:00100000:3.0:1789760788.009112:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718123921254 time 718123660838 state 0x1 ref 1 00000100:00100000:3.0:1789760788.009146:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718123921254 time 718133751633 state 0x0 ref 1 00000100:00100000:3.0:1789760788.009158:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.009210:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79392768, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25dd0 00010000:00010000:3.0:1789760788.009219:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.009233:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.009238:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00000100:00080000:1.0:1789760788.009341:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789760788.009368:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.009385:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.016527:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986690 is committed 00002000:00010000:2.0:1789760788.016916:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79392768 -> 79396864 00002000:00010000:2.0:1789760788.016922:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155088 -> 155096 00010000:00010000:2.0:1789760788.016936:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.016948:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.016955:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.016961:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.016966:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.016974:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.017648:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.017657:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79392768->79396863] (req 79392768->79396863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.017817:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8655us (8960us total) trans 4294986690 rc 0/0 00000100:00100000:2.0:1789760788.017832:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19394 00000100:00100000:2.0:1789760788.017842:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718123921254 time 718133751633 state 0x2 ref 0 00000100:00100000:2.0:1789760788.027598:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.027682:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870720 00000100:00100000:3.0:1789760788.027692:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870720 00000100:00100000:3.0:1789760788.027751:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718123921254 time 718133751633 state 0x0 ref 1 00000100:00100000:3.0:1789760788.027767:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718134012049 time 718133751633 state 0x1 ref 1 00000100:00100000:3.0:1789760788.027800:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718134012049 time 718152406437 state 0x0 ref 1 00000100:00100000:3.0:1789760788.027815:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.027863:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79396864, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25dd8 00010000:00010000:3.0:1789760788.027872:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.027888:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.027892:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.027898:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.027911:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.032421:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986691 is committed 00002000:00010000:0.0:1789760788.032542:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79396864 -> 79400960 00002000:00010000:0.0:1789760788.032548:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155096 -> 155104 00010000:00010000:0.0:1789760788.032561:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.032572:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.032581:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.032595:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.032599:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.032608:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.032627:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.032634:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1a11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79396864->79400959] (req 79396864->79400959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.032752:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4937us (5159us total) trans 4294986691 rc 0/0 00000100:00100000:0.0:1789760788.032759:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19395 00000100:00100000:0.0:1789760788.032767:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718134012049 time 718152406437 state 0x2 ref 0 00000100:00100000:1.0:1789760788.035634:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.035785:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870848 00000100:00100000:0.0:1789760788.035796:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870848 00000100:00100000:0.0:1789760788.035851:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718134012049 time 718152406437 state 0x0 ref 1 00000100:00100000:0.0:1789760788.035867:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718152666853 time 718152406437 state 0x1 ref 1 00000100:00100000:0.0:1789760788.035899:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718152666853 time 718160505636 state 0x0 ref 1 00000100:00100000:0.0:1789760788.035916:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.035961:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79400960, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25de0 00010000:00010000:0.0:1789760788.035971:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.035987:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.035991:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.035997:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.036010:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.041234:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986692 is committed 00002000:00010000:0.0:1789760788.041715:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79400960 -> 79405056 00002000:00010000:0.0:1789760788.041720:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155104 -> 155112 00010000:00010000:0.0:1789760788.041733:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.041745:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.041756:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.041764:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.041769:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.041779:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.041914:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.041923:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79400960->79405055] (req 79400960->79405055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.042017:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503870848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6098us (6448us total) trans 4294986692 rc 0/0 00000100:00100000:0.0:1789760788.042027:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19396 00000100:00100000:0.0:1789760788.042038:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718152666853 time 718160505636 state 0x2 ref 0 00000100:00100000:1.0:1789760788.046613:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.046893:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503870976 00000100:00100000:0.0:1789760788.046904:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503870976 00000100:00100000:0.0:1789760788.046959:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718152666853 time 718160505636 state 0x0 ref 1 00000100:00100000:0.0:1789760788.046975:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718160766052 time 718160505636 state 0x1 ref 1 00000100:00100000:0.0:1789760788.047000:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718160766052 time 718171606555 state 0x0 ref 1 00000100:00100000:0.0:1789760788.047014:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.047060:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79405056, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25de8 00010000:00010000:0.0:1789760788.047070:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.047085:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.047089:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.047096:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.047110:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.054162:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986693 is committed 00002000:00010000:1.0:1789760788.054657:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79405056 -> 79409152 00002000:00010000:1.0:1789760788.054662:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155112 -> 155120 00010000:00010000:1.0:1789760788.054676:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.054688:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.054697:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.054705:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.054709:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.054719:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.054738:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.054746:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1a1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79405056->79409151] (req 79405056->79409151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.054889:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503870976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7872us (8280us total) trans 4294986693 rc 0/0 00000100:00100000:1.0:1789760788.054970:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19397 00000100:00100000:1.0:1789760788.054981:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718160766052 time 718171606555 state 0x2 ref 0 00000100:00100000:2.0:1789760788.057881:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.058062:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503871104 00000100:00100000:3.0:1789760788.058071:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503871104 00000100:00100000:3.0:1789760788.058124:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718160766052 time 718171606555 state 0x0 ref 1 00000100:00100000:3.0:1789760788.058139:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718171866971 time 718171606555 state 0x1 ref 1 00000100:00100000:3.0:1789760788.058171:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718171866971 time 718182777243 state 0x0 ref 1 00000100:00100000:3.0:1789760788.058185:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503871104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.058228:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79409152, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25df0 00010000:00010000:3.0:1789760788.058237:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.058368:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.058373:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.058379:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.058423:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.062971:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986694 is committed 00002000:00010000:3.0:1789760788.063083:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79409152 -> 79413248 00002000:00010000:3.0:1789760788.063088:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155120 -> 155128 00010000:00010000:3.0:1789760788.063101:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.063112:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.063121:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.063129:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.063134:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.063143:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.063162:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.063170:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79409152->79413247] (req 79409152->79413247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.063275:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503871104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5087us (5399us total) trans 4294986694 rc 0/0 00000100:00100000:3.0:1789760788.063286:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19398 00000100:00100000:3.0:1789760788.063296:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718171866971 time 718182777243 state 0x2 ref 0 00000100:00100000:3.0:1789760788.067939:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.067999:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503871232 00000100:00100000:1.0:1789760788.068010:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503871232 00000100:00100000:1.0:1789760788.068064:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718171866971 time 718182777243 state 0x0 ref 1 00000100:00100000:1.0:1789760788.068079:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718183037659 time 718182777243 state 0x1 ref 1 00000100:00100000:1.0:1789760788.068171:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718183037659 time 718192776780 state 0x0 ref 1 00000100:00100000:1.0:1789760788.068184:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.068231:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79413248, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25df8 00010000:00010000:1.0:1789760788.068240:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.068266:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.068270:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.068277:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.068290:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.074989:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986695 is committed 00002000:00010000:1.0:1789760788.075238:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79413248 -> 79417344 00002000:00010000:1.0:1789760788.075634:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155128 -> 155136 00010000:00010000:1.0:1789760788.075650:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.075662:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.075677:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.075684:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.075689:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.075698:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.075719:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.075728:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79413248->79417343] (req 79413248->79417343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.075875:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7689us (7941us total) trans 4294986695 rc 0/0 00000100:00100000:1.0:1789760788.075887:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19399 00000100:00100000:1.0:1789760788.075896:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718183037659 time 718192776780 state 0x2 ref 0 00000100:00100000:1.0:1789760788.079858:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.079945:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503871360 00000100:00100000:0.0:1789760788.079954:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503871360 00000100:00100000:0.0:1789760788.080007:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718183037659 time 718192776780 state 0x0 ref 1 00000100:00100000:0.0:1789760788.080023:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718193037196 time 718192776780 state 0x1 ref 1 00000100:00100000:0.0:1789760788.080055:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718193037196 time 718204661334 state 0x0 ref 1 00000100:00100000:0.0:1789760788.080068:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.080111:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79417344, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e00 00010000:00010000:0.0:1789760788.080120:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.080139:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.080144:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.080150:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.080163:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.085470:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986696 is committed 00002000:00010000:0.0:1789760788.086059:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79417344 -> 79421440 00002000:00010000:0.0:1789760788.086064:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155136 -> 155144 00010000:00010000:0.0:1789760788.086077:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.086089:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.086098:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.086105:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.086110:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.086118:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.086143:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.086151:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1a34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79417344->79421439] (req 79417344->79421439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.086533:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6452us (6669us total) trans 4294986696 rc 0/0 00000100:00100000:0.0:1789760788.086544:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19400 00000100:00100000:0.0:1789760788.086554:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718193037196 time 718204661334 state 0x2 ref 0 00000100:00100000:2.0:1789760788.094968:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.098494:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503871488 00000100:00100000:0.0:1789760788.098509:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503871488 00000100:00100000:1.0:1789760788.098832:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718193037196 time 718204661334 state 0x0 ref 1 00000100:00100000:1.0:1789760788.098848:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718204921750 time 718204661334 state 0x1 ref 1 00000100:00100000:1.0:1789760788.098922:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718204921750 time 718223527465 state 0x0 ref 1 00000100:00100000:1.0:1789760788.098937:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503871488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.098986:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79421440, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e08 00010000:00010000:1.0:1789760788.098995:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.099010:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.099015:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.099021:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.099035:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.103977:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986697 is committed 00002000:00010000:1.0:1789760788.105828:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79421440 -> 79425536 00002000:00010000:1.0:1789760788.105837:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155144 -> 155152 00010000:00010000:1.0:1789760788.105851:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.105862:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.105869:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.105882:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.105887:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.105958:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.105981:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.105989:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79421440->79425535] (req 79421440->79425535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.106143:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503871488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7205us (11181us total) trans 4294986697 rc 0/0 00000100:00100000:1.0:1789760788.106153:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19401 00000100:00100000:1.0:1789760788.106163:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718204921750 time 718223527465 state 0x2 ref 0 00000100:00100000:3.0:1789760788.108870:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.108926:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503871616 00000100:00100000:2.0:1789760788.108934:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503871616 00000100:00100000:2.0:1789760788.108991:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718204921750 time 718223527465 state 0x0 ref 1 00000100:00100000:2.0:1789760788.109004:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718223787881 time 718223527465 state 0x1 ref 1 00000100:00100000:2.0:1789760788.109082:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718223787881 time 718233688117 state 0x0 ref 1 00000100:00100000:2.0:1789760788.109095:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.109138:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79425536, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e10 00010000:00010000:2.0:1789760788.109146:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.109160:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.109163:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.109169:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.109181:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.114444:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986698 is committed 00002000:00010000:2.0:1789760788.114585:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79425536 -> 79429632 00002000:00010000:2.0:1789760788.114591:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155152 -> 155160 00010000:00010000:2.0:1789760788.114603:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.114616:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.114624:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.114631:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.114636:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.114646:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.114666:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.114673:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1a42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79425536->79429631] (req 79425536->79429631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.114764:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5666us (5895us total) trans 4294986698 rc 0/0 00000100:00100000:2.0:1789760788.114773:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19402 00000100:00100000:2.0:1789760788.114782:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718223787881 time 718233688117 state 0x2 ref 0 00000100:00100000:2.0:1789760788.120456:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.120970:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503871744 00000100:00100000:0.0:1789760788.120981:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503871744 00000100:00100000:0.0:1789760788.121038:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718223787881 time 718233688117 state 0x0 ref 1 00000100:00100000:0.0:1789760788.121053:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718233948533 time 718233688117 state 0x1 ref 1 00000100:00100000:0.0:1789760788.121089:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718233948533 time 718245695389 state 0x0 ref 1 00000100:00100000:0.0:1789760788.121102:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.121151:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79429632, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e18 00010000:00010000:0.0:1789760788.121171:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.121192:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.121197:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.121203:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.121217:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.127348:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986699 is committed 00002000:00010000:0.0:1789760788.128011:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79429632 -> 79433728 00002000:00010000:0.0:1789760788.128017:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155160 -> 155168 00010000:00010000:0.0:1789760788.128030:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.128043:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.128051:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.128059:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.128064:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.128073:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.128098:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.128104:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79429632->79433727] (req 79429632->79433727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.128173:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503871744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7072us (8322us total) trans 4294986699 rc 0/0 00000100:00100000:0.0:1789760788.128180:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19403 00000100:00100000:0.0:1789760788.128187:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718233948533 time 718245695389 state 0x2 ref 0 00000100:00100000:0.0:1789760788.135225:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.135344:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503871872 00000100:00100000:1.0:1789760788.135356:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503871872 00000100:00100000:1.0:1789760788.135448:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718233948533 time 718245695389 state 0x0 ref 1 00000100:00100000:1.0:1789760788.135463:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718245955805 time 718245695389 state 0x1 ref 1 00000100:00100000:1.0:1789760788.135493:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718245955805 time 718260099965 state 0x0 ref 1 00000100:00100000:1.0:1789760788.135508:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503871872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.135576:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79433728, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e20 00010000:00010000:1.0:1789760788.135585:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.135601:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.135605:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.135612:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.135625:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.141265:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986700 is committed 00002000:00010000:1.0:1789760788.141378:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79433728 -> 79437824 00002000:00010000:1.0:1789760788.141384:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155168 -> 155176 00010000:00010000:1.0:1789760788.141419:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141433:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141441:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141447:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.141452:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141461:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141478:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141741:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79433728->79437823] (req 79433728->79437823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.141828:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503871872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6317us (6607us total) trans 4294986700 rc 0/0 00000100:00100000:1.0:1789760788.141839:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19404 00000100:00100000:1.0:1789760788.141851:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718245955805 time 718260099965 state 0x2 ref 0 00000100:00100000:3.0:1789760788.145834:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.145973:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872000 00000100:00100000:1.0:1789760788.145983:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872000 00000100:00100000:1.0:1789760788.146032:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718245955805 time 718260099965 state 0x0 ref 1 00000100:00100000:1.0:1789760788.146046:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718260360381 time 718260099965 state 0x1 ref 1 00000100:00100000:1.0:1789760788.146079:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718260360381 time 718270685661 state 0x0 ref 1 00000100:00100000:1.0:1789760788.146090:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503872000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.146134:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79437824, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e28 00010000:00010000:1.0:1789760788.146143:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.146157:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.146162:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.146168:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.146181:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.151921:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986701 is committed 00002000:00010000:0.0:1789760788.152104:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79437824 -> 79441920 00002000:00010000:0.0:1789760788.152110:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155176 -> 155184 00010000:00010000:0.0:1789760788.152124:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.152136:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.152145:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.152153:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.152158:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.152167:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.152185:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.152194:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1a57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79437824->79441919] (req 79437824->79441919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.152710:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503872000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6615us (6887us total) trans 4294986701 rc 0/0 00000100:00100000:0.0:1789760788.152722:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19405 00000100:00100000:0.0:1789760788.152732:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718260360381 time 718270685661 state 0x2 ref 0 00000100:00100000:2.0:1789760788.160497:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.160797:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872128 00000100:00100000:3.0:1789760788.160808:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872128 00000100:00100000:3.0:1789760788.160865:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718260360381 time 718270685661 state 0x0 ref 1 00000100:00100000:3.0:1789760788.160880:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718270946077 time 718270685661 state 0x1 ref 1 00000100:00100000:3.0:1789760788.160913:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718270946077 time 718285519403 state 0x0 ref 1 00000100:00100000:3.0:1789760788.160926:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503872128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.160982:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79441920, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e30 00010000:00010000:3.0:1789760788.160991:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.161006:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.161010:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.161016:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.161029:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.165734:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986702 is committed 00002000:00010000:3.0:1789760788.165848:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79441920 -> 79446016 00002000:00010000:3.0:1789760788.165854:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155184 -> 155192 00010000:00010000:3.0:1789760788.165866:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.165877:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.165885:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.165898:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.165903:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.165913:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.165933:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.165941:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1a5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79441920->79446015] (req 79441920->79446015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.166060:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503872128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5131us (5567us total) trans 4294986702 rc 0/0 00000100:00100000:3.0:1789760788.166070:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19406 00000100:00100000:3.0:1789760788.166080:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718270946077 time 718285519403 state 0x2 ref 0 00000100:00100000:1.0:1789760788.169466:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.169638:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872256 00000100:00100000:0.0:1789760788.169648:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872256 00000100:00100000:0.0:1789760788.169706:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718270946077 time 718285519403 state 0x0 ref 1 00000100:00100000:0.0:1789760788.169721:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718285779819 time 718285519403 state 0x1 ref 1 00000100:00100000:0.0:1789760788.169754:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718285779819 time 718294360479 state 0x0 ref 1 00000100:00100000:0.0:1789760788.169767:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503872256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.169814:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79446016, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e38 00010000:00010000:0.0:1789760788.169824:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.169841:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.169845:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.169852:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.169865:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.174822:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986703 is committed 00002000:00010000:0.0:1789760788.174943:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79446016 -> 79450112 00002000:00010000:0.0:1789760788.174949:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155192 -> 155200 00010000:00010000:0.0:1789760788.174962:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.174986:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.174997:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.175004:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.175010:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.175020:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.175042:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.175050:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79446016->79450111] (req 79446016->79450111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.175146:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503872256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5377us (5685us total) trans 4294986703 rc 0/0 00000100:00100000:0.0:1789760788.175156:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19407 00000100:00100000:0.0:1789760788.175166:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718285779819 time 718294360479 state 0x2 ref 0 00000100:00100000:1.0:1789760788.178899:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.178993:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872384 00000100:00100000:0.0:1789760788.179003:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872384 00000100:00100000:0.0:1789760788.179054:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718285779819 time 718294360479 state 0x0 ref 1 00000100:00100000:0.0:1789760788.179068:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718294620895 time 718294360479 state 0x1 ref 1 00000100:00100000:0.0:1789760788.179098:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718294620895 time 718303705128 state 0x0 ref 1 00000100:00100000:0.0:1789760788.179111:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503872384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.179155:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79450112, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e40 00010000:00010000:0.0:1789760788.179162:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.179176:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.179181:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.179187:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.179200:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.184563:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986704 is committed 00002000:00010000:0.0:1789760788.184676:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79450112 -> 79454208 00002000:00010000:0.0:1789760788.184681:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155200 -> 155208 00010000:00010000:0.0:1789760788.184694:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.184706:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.184715:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.184722:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.184726:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.184735:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.184754:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.184766:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79450112->79454207] (req 79450112->79454207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.184859:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503872384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5745us (5965us total) trans 4294986704 rc 0/0 00000100:00100000:0.0:1789760788.184869:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19408 00000100:00100000:0.0:1789760788.184878:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718294620895 time 718303705128 state 0x2 ref 0 00000100:00100000:2.0:1789760788.199213:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.199434:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872512 00000100:00100000:1.0:1789760788.199444:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872512 00000100:00100000:1.0:1789760788.199497:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718294620895 time 718303705128 state 0x0 ref 1 00000100:00100000:1.0:1789760788.199512:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718303965544 time 718303705128 state 0x1 ref 1 00000100:00100000:1.0:1789760788.199550:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718303965544 time 718324156017 state 0x0 ref 1 00000100:00100000:1.0:1789760788.199561:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503872512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.199605:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79454208, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e48 00010000:00010000:1.0:1789760788.199614:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.199634:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.199638:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.199644:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.199657:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.204903:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986705 is committed 00002000:00010000:1.0:1789760788.210207:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79454208 -> 79458304 00002000:00010000:1.0:1789760788.210218:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155208 -> 155216 00010000:00010000:1.0:1789760788.210231:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.210245:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.210253:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.210260:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.210265:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.210274:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.210297:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.210305:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f1a73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79454208->79458303] (req 79454208->79458303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.210583:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503872512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11018us (11374us total) trans 4294986705 rc 0/0 00000100:00100000:1.0:1789760788.210594:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19409 00000100:00100000:1.0:1789760788.210604:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718303965544 time 718324156017 state 0x2 ref 0 00000100:00100000:1.0:1789760788.214204:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.214931:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872640 00000100:00100000:3.0:1789760788.214942:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872640 00000100:00100000:0.0:1789760788.217014:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718303965544 time 718324156017 state 0x0 ref 1 00000100:00100000:0.0:1789760788.217032:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718324416433 time 718324156017 state 0x1 ref 1 00000100:00100000:0.0:1789760788.217073:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718324416433 time 718341680072 state 0x0 ref 1 00000100:00100000:0.0:1789760788.217088:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503872640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.217135:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79458304, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e50 00010000:00010000:0.0:1789760788.217145:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.217160:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.217164:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.217170:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.217182:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.224068:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986706 is committed 00002000:00010000:0.0:1789760788.224185:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79458304 -> 79462400 00002000:00010000:0.0:1789760788.224190:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155216 -> 155224 00010000:00010000:0.0:1789760788.224202:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.224214:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.224222:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.224230:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.224234:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.224243:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.224769:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.224778:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1a7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79458304->79462399] (req 79458304->79462399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.225027:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503872640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7937us (10827us total) trans 4294986706 rc 0/0 00000100:00100000:0.0:1789760788.225038:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19410 00000100:00100000:0.0:1789760788.225047:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718324416433 time 718341680072 state 0x2 ref 0 00000100:00100000:1.0:1789760788.230106:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.230188:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872768 00000100:00100000:0.0:1789760788.230198:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872768 00000100:00100000:0.0:1789760788.230313:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718324416433 time 718341680072 state 0x0 ref 1 00000100:00100000:0.0:1789760788.230329:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718341940488 time 718341680072 state 0x1 ref 1 00000100:00100000:0.0:1789760788.230410:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718341940488 time 718354994642 state 0x0 ref 1 00000100:00100000:0.0:1789760788.230424:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503872768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.230481:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79462400, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e58 00010000:00010000:0.0:1789760788.230490:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.230506:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.230510:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.230516:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.230538:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.234483:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986707 is committed 00002000:00010000:0.0:1789760788.234602:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79462400 -> 79466496 00002000:00010000:0.0:1789760788.234607:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155224 -> 155232 00010000:00010000:0.0:1789760788.234619:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.234630:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.234638:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.234645:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.234650:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.234659:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.234677:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.234685:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79462400->79466495] (req 79462400->79466495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.234772:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503872768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4346us (4671us total) trans 4294986707 rc 0/0 00000100:00100000:0.0:1789760788.234786:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19411 00000100:00100000:0.0:1789760788.234796:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718341940488 time 718354994642 state 0x2 ref 0 00000100:00100000:2.0:1789760788.240727:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.244550:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503872896 00000100:00100000:1.0:1789760788.244628:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503872896 00000100:00100000:1.0:1789760788.244703:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718341940488 time 718354994642 state 0x0 ref 1 00000100:00100000:1.0:1789760788.244720:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718355255058 time 718354994642 state 0x1 ref 1 00000100:00100000:1.0:1789760788.244817:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718355255058 time 718369421616 state 0x0 ref 1 00000100:00100000:1.0:1789760788.244831:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503872896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.244881:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79466496, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e60 00010000:00010000:1.0:1789760788.244890:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.244910:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.244915:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.244921:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.244935:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.252780:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986708 is committed 00002000:00010000:0.0:1789760788.253343:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79466496 -> 79470592 00002000:00010000:0.0:1789760788.253349:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155232 -> 155240 00010000:00010000:0.0:1789760788.253364:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.253377:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.253385:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.253433:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.253438:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.253448:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.253469:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.253478:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1a88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79466496->79470591] (req 79466496->79470591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.253573:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503872896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8739us (12852us total) trans 4294986708 rc 0/0 00000100:00100000:0.0:1789760788.253587:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19412 00000100:00100000:0.0:1789760788.253596:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718355255058 time 718369421616 state 0x2 ref 0 00000100:00100000:1.0:1789760788.261482:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.262477:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873024 00000100:00100000:0.0:1789760788.262489:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873024 00000100:00100000:0.0:1789760788.262555:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718355255058 time 718369421616 state 0x0 ref 1 00000100:00100000:0.0:1789760788.262571:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718369682032 time 718369421616 state 0x1 ref 1 00000100:00100000:0.0:1789760788.262604:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718369682032 time 718387210941 state 0x0 ref 1 00000100:00100000:0.0:1789760788.262617:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503873024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.262667:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79470592, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e68 00010000:00010000:0.0:1789760788.262679:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.262696:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.262700:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.262707:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.262732:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.266915:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986709 is committed 00002000:00010000:0.0:1789760788.267209:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79470592 -> 79474688 00002000:00010000:0.0:1789760788.267214:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155240 -> 155248 00010000:00010000:0.0:1789760788.267229:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.267250:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.267260:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.267267:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.267272:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.267281:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.267301:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.267308:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1a8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79470592->79474687] (req 79470592->79474687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.267447:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503873024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4826us (5968us total) trans 4294986709 rc 0/0 00000100:00100000:0.0:1789760788.267457:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19413 00000100:00100000:0.0:1789760788.267466:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718369682032 time 718387210941 state 0x2 ref 0 00000100:00100000:1.0:1789760788.273876:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.273990:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873152 00000100:00100000:0.0:1789760788.274002:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873152 00000100:00100000:2.0:1789760788.274147:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718369682032 time 718387210941 state 0x0 ref 1 00000100:00100000:2.0:1789760788.274162:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718387471357 time 718387210941 state 0x1 ref 1 00000100:00100000:2.0:1789760788.274242:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718387471357 time 718398846803 state 0x0 ref 1 00000100:00100000:2.0:1789760788.274267:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503873152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.274313:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79474688, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e70 00010000:00010000:2.0:1789760788.274322:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.274336:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.274340:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.274347:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.274361:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.278865:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986710 is committed 00002000:00010000:2.0:1789760788.291748:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79474688 -> 79478784 00002000:00010000:2.0:1789760788.291758:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155248 -> 155256 00010000:00010000:2.0:1789760788.291771:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.291784:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.291793:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.291800:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.291805:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.291813:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.291833:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.291846:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1a96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79474688->79478783] (req 79474688->79478783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.291951:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503873152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 17681us (18077us total) trans 4294986710 rc 0/0 00000100:00100000:2.0:1789760788.291961:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19414 00000100:00100000:2.0:1789760788.291972:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718387471357 time 718398846803 state 0x2 ref 0 00000100:00100000:2.0:1789760788.303978:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.304101:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873280 00000100:00100000:1.0:1789760788.304111:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873280 00000100:00100000:1.0:1789760788.304162:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718387471357 time 718398846803 state 0x0 ref 1 00000100:00100000:1.0:1789760788.304177:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718399107219 time 718398846803 state 0x1 ref 1 00000100:00100000:1.0:1789760788.304204:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718399107219 time 718428810822 state 0x0 ref 1 00000100:00100000:1.0:1789760788.304215:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503873280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.304274:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79478784, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e78 00010000:00010000:1.0:1789760788.304284:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304304:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.304308:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.304314:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304327:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.309750:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986711 is committed 00002000:00010000:1.0:1789760788.316287:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79478784 -> 79482880 00002000:00010000:1.0:1789760788.316297:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155256 -> 155264 00010000:00010000:1.0:1789760788.316312:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316325:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316334:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316342:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.316346:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316354:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316380:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316387:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1a9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79478784->79482879] (req 79478784->79482879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.316533:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503873280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12306us (12550us total) trans 4294986711 rc 0/0 00000100:00100000:1.0:1789760788.316542:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19415 00000100:00100000:1.0:1789760788.316883:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718399107219 time 718428810822 state 0x2 ref 0 00000100:00100000:1.0:1789760788.322269:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.322353:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873408 00000100:00100000:0.0:1789760788.322364:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873408 00000100:00100000:0.0:1789760788.322457:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718399107219 time 718428810822 state 0x0 ref 1 00000100:00100000:0.0:1789760788.322472:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718429071238 time 718428810822 state 0x1 ref 1 00000100:00100000:0.0:1789760788.322492:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718429071238 time 718447099012 state 0x0 ref 1 00000100:00100000:0.0:1789760788.322504:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503873408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.322565:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79482880, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e80 00010000:00010000:0.0:1789760788.322575:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.322589:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.322593:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.322599:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.322612:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.328833:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986712 is committed 00002000:00010000:0.0:1789760788.328989:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79482880 -> 79486976 00002000:00010000:0.0:1789760788.328994:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155264 -> 155272 00010000:00010000:0.0:1789760788.329007:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.329020:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.329028:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.329035:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.329040:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.329049:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.329067:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.329076:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1aa4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79482880->79486975] (req 79482880->79486975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.329166:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503873408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6659us (6901us total) trans 4294986712 rc 0/0 00000100:00100000:0.0:1789760788.329177:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19416 00000100:00100000:0.0:1789760788.329186:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718429071238 time 718447099012 state 0x2 ref 0 00000100:00100000:2.0:1789760788.331890:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.332039:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873536 00000100:00100000:0.0:1789760788.332049:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873536 00000100:00100000:0.0:1789760788.332101:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718429071238 time 718447099012 state 0x0 ref 1 00000100:00100000:0.0:1789760788.332116:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718447359428 time 718447099012 state 0x1 ref 1 00000100:00100000:0.0:1789760788.332147:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718447359428 time 718456753512 state 0x0 ref 1 00000100:00100000:0.0:1789760788.332159:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503873536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.332204:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79486976, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e88 00010000:00010000:0.0:1789760788.332212:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.332226:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.332230:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.332236:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.332250:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.342842:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986713 is committed 00002000:00010000:0.0:1789760788.342991:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79486976 -> 79491072 00002000:00010000:0.0:1789760788.342996:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155272 -> 155280 00010000:00010000:0.0:1789760788.343008:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.343019:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.343034:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.343042:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.343046:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.343055:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.343074:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.343081:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1aab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79486976->79491071] (req 79486976->79491071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.343179:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503873536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11017us (11292us total) trans 4294986713 rc 0/0 00000100:00100000:0.0:1789760788.343188:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19417 00000100:00100000:0.0:1789760788.343197:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718447359428 time 718456753512 state 0x2 ref 0 00000100:00100000:2.0:1789760788.349050:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.349155:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873664 00000100:00100000:0.0:1789760788.349166:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873664 00000100:00100000:0.0:1789760788.349215:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718447359428 time 718456753512 state 0x0 ref 1 00000100:00100000:0.0:1789760788.349229:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718457013928 time 718456753512 state 0x1 ref 1 00000100:00100000:0.0:1789760788.349351:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718457013928 time 718473956734 state 0x0 ref 1 00000100:00100000:0.0:1789760788.349363:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503873664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.349442:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79491072, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e90 00010000:00010000:0.0:1789760788.349451:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.349466:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.349471:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.349477:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.349491:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.355046:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986714 is committed 00002000:00010000:0.0:1789760788.355371:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79491072 -> 79495168 00002000:00010000:0.0:1789760788.355377:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155280 -> 155288 00010000:00010000:0.0:1789760788.355410:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.355431:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.355440:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.355454:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.355459:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.355468:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.355488:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.355496:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ab2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79491072->79495167] (req 79491072->79495167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.355615:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503873664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6248us (6569us total) trans 4294986714 rc 0/0 00000100:00100000:0.0:1789760788.355625:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19418 00000100:00100000:0.0:1789760788.355635:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718457013928 time 718473956734 state 0x2 ref 0 00000100:00100000:1.0:1789760788.359736:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.363997:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873792 00000100:00100000:0.0:1789760788.364012:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873792 00000100:00100000:0.0:1789760788.364076:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718457013928 time 718473956734 state 0x0 ref 1 00000100:00100000:0.0:1789760788.364091:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718474217150 time 718473956734 state 0x1 ref 1 00000100:00100000:0.0:1789760788.364116:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718474217150 time 718488723015 state 0x0 ref 1 00000100:00100000:0.0:1789760788.364133:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503873792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.364182:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79495168, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25e98 00010000:00010000:0.0:1789760788.364191:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.364207:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.364211:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.364217:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.364231:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.372470:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986715 is committed 00002000:00010000:1.0:1789760788.373051:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79495168 -> 79499264 00002000:00010000:1.0:1789760788.373057:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155288 -> 155296 00010000:00010000:1.0:1789760788.373069:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.373082:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.373092:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.373099:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.373105:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.373114:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.373134:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.373141:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1ab9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79495168->79499263] (req 79495168->79499263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.373324:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503873792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9190us (13593us total) trans 4294986715 rc 0/0 00000100:00100000:1.0:1789760788.373335:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19419 00000100:00100000:1.0:1789760788.373345:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718474217150 time 718488723015 state 0x2 ref 0 00000100:00100000:2.0:1789760788.379007:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.379100:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503873920 00000100:00100000:0.0:1789760788.379110:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503873920 00000100:00100000:0.0:1789760788.379166:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718474217150 time 718488723015 state 0x0 ref 1 00000100:00100000:0.0:1789760788.379181:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718488983431 time 718488723015 state 0x1 ref 1 00000100:00100000:0.0:1789760788.379308:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718488983431 time 718503912572 state 0x0 ref 1 00000100:00100000:0.0:1789760788.379322:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503873920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.379509:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79499264, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ea0 00010000:00010000:0.0:1789760788.379646:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.379668:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.379672:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.379680:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.379694:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.387256:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986716 is committed 00002000:00010000:2.0:1789760788.387692:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79499264 -> 79503360 00002000:00010000:2.0:1789760788.387697:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155296 -> 155304 00010000:00010000:2.0:1789760788.387711:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387722:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387730:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387737:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.387742:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387751:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387771:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387779:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ac0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79499264->79503359] (req 79499264->79503359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.387923:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503873920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8598us (8922us total) trans 4294986716 rc 0/0 00000100:00100000:2.0:1789760788.387934:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19420 00000100:00100000:2.0:1789760788.387943:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718488983431 time 718503912572 state 0x2 ref 0 00000100:00100000:2.0:1789760788.400256:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.400724:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874048 00000100:00100000:1.0:1789760788.400736:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874048 00000100:00100000:1.0:1789760788.400793:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718488983431 time 718503912572 state 0x0 ref 1 00000100:00100000:1.0:1789760788.400810:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718504172988 time 718503912572 state 0x1 ref 1 00000100:00100000:1.0:1789760788.400852:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718504172988 time 718525457516 state 0x0 ref 1 00000100:00100000:1.0:1789760788.400865:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.400918:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79503360, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ea8 00010000:00010000:1.0:1789760788.400933:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.400949:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.400954:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.400961:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.400976:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.409607:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986717 is committed 00002000:00010000:1.0:1789760788.409785:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79503360 -> 79507456 00002000:00010000:1.0:1789760788.409791:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155304 -> 155312 00010000:00010000:1.0:1789760788.409804:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.409815:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.409824:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.409830:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.409835:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.409850:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.409868:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.409877:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1ac7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79503360->79507455] (req 79503360->79507455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.409969:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9100us (9733us total) trans 4294986717 rc 0/0 00000100:00100000:1.0:1789760788.409980:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19421 00000100:00100000:1.0:1789760788.409990:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718504172988 time 718525457516 state 0x2 ref 0 00000100:00100000:1.0:1789760788.418382:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.418501:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874176 00000100:00100000:2.0:1789760788.418512:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874176 00000100:00100000:2.0:1789760788.418585:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718504172988 time 718525457516 state 0x0 ref 1 00000100:00100000:2.0:1789760788.418600:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718525717932 time 718525457516 state 0x1 ref 1 00000100:00100000:2.0:1789760788.418630:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718525717932 time 718543237160 state 0x0 ref 1 00000100:00100000:2.0:1789760788.418643:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.418694:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79507456, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25eb0 00010000:00010000:2.0:1789760788.418703:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.418729:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.418734:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.418741:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.418755:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.424610:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986718 is committed 00002000:00010000:2.0:1789760788.424871:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79507456 -> 79511552 00002000:00010000:2.0:1789760788.424876:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155312 -> 155320 00010000:00010000:2.0:1789760788.424891:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.424903:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.424912:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.424919:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.424924:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.424938:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.424956:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.424965:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ace lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79507456->79511551] (req 79507456->79511551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.425059:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6412us (6681us total) trans 4294986718 rc 0/0 00000100:00100000:2.0:1789760788.425069:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19422 00000100:00100000:2.0:1789760788.425078:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718525717932 time 718543237160 state 0x2 ref 0 00000100:00100000:2.0:1789760788.428838:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.428966:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874304 00000100:00100000:1.0:1789760788.428977:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874304 00000100:00100000:1.0:1789760788.429031:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718525717932 time 718543237160 state 0x0 ref 1 00000100:00100000:1.0:1789760788.429046:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718543497576 time 718543237160 state 0x1 ref 1 00000100:00100000:1.0:1789760788.429079:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718543497576 time 718553685604 state 0x0 ref 1 00000100:00100000:1.0:1789760788.429093:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.429160:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79511552, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25eb8 00010000:00010000:1.0:1789760788.429169:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.429184:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.429189:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.429195:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.429208:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.438588:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986719 is committed 00002000:00010000:2.0:1789760788.438751:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79511552 -> 79515648 00002000:00010000:2.0:1789760788.438757:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155320 -> 155328 00010000:00010000:2.0:1789760788.438771:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.438784:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.438793:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.438805:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.438810:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.438819:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.438838:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.438847:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f1ad5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79511552->79515647] (req 79511552->79515647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.438938:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9843us (10106us total) trans 4294986719 rc 0/0 00000100:00100000:2.0:1789760788.438949:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19423 00000100:00100000:2.0:1789760788.438959:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718543497576 time 718553685604 state 0x2 ref 0 00000100:00100000:1.0:1789760788.444876:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.444976:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874432 00000100:00100000:2.0:1789760788.444986:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874432 00000100:00100000:2.0:1789760788.445039:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718543497576 time 718553685604 state 0x0 ref 1 00000100:00100000:2.0:1789760788.445054:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718553946020 time 718553685604 state 0x1 ref 1 00000100:00100000:2.0:1789760788.445088:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718553946020 time 718569695163 state 0x0 ref 1 00000100:00100000:2.0:1789760788.445104:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503874432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.445160:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79515648, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ec0 00010000:00010000:2.0:1789760788.445169:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.445184:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.445188:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.445194:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.445207:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.453443:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986720 is committed 00002000:00010000:1.0:1789760788.453607:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79515648 -> 79519744 00002000:00010000:1.0:1789760788.453612:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155328 -> 155336 00010000:00010000:1.0:1789760788.453624:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.453638:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.453646:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.453653:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.453658:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.453672:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.453691:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.453698:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1adc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79515648->79519743] (req 79515648->79519743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.453787:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503874432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8681us (8916us total) trans 4294986720 rc 0/0 00000100:00100000:1.0:1789760788.453796:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19424 00000100:00100000:1.0:1789760788.453806:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718553946020 time 718569695163 state 0x2 ref 0 00000100:00100000:1.0:1789760788.458767:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.458851:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874560 00000100:00100000:0.0:1789760788.458861:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874560 00000100:00100000:0.0:1789760788.458927:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718553946020 time 718569695163 state 0x0 ref 1 00000100:00100000:0.0:1789760788.458943:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718569955579 time 718569695163 state 0x1 ref 1 00000100:00100000:0.0:1789760788.458975:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718569955579 time 718583581607 state 0x0 ref 1 00000100:00100000:0.0:1789760788.458987:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.459038:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79519744, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ec8 00010000:00010000:0.0:1789760788.459047:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.459063:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.459068:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.459074:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.459094:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.466587:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986721 is committed 00002000:00010000:0.0:1789760788.466698:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79519744 -> 79523840 00002000:00010000:0.0:1789760788.466703:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155336 -> 155344 00010000:00010000:0.0:1789760788.466718:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.466732:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.466741:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.466748:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.466752:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.466761:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.466783:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.466791:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1ae3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79519744->79523839] (req 79519744->79523839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.466899:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503874560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7908us (8136us total) trans 4294986721 rc 0/0 00000100:00100000:0.0:1789760788.466910:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19425 00000100:00100000:0.0:1789760788.466922:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718569955579 time 718583581607 state 0x2 ref 0 00000100:00100000:2.0:1789760788.473050:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.477129:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874688 00000100:00100000:0.0:1789760788.477144:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874688 00000100:00100000:0.0:1789760788.477208:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718569955579 time 718583581607 state 0x0 ref 1 00000100:00100000:0.0:1789760788.477225:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718583842023 time 718583581607 state 0x1 ref 1 00000100:00100000:1.0:1789760788.477312:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718583842023 time 718601914803 state 0x0 ref 1 00000100:00100000:1.0:1789760788.477329:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503874688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.477375:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79523840, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ed0 00010000:00010000:1.0:1789760788.477385:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.477421:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.477426:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.477433:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.477448:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.485859:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986722 is committed 00002000:00010000:1.0:1789760788.485973:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79523840 -> 79527936 00002000:00010000:1.0:1789760788.485977:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155344 -> 155352 00010000:00010000:1.0:1789760788.485989:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.486001:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.486009:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.486016:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.486021:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.486030:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.486048:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.486062:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1aea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79523840->79527935] (req 79523840->79527935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.486155:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503874688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8825us (13111us total) trans 4294986722 rc 0/0 00000100:00100000:1.0:1789760788.486166:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19426 00000100:00100000:1.0:1789760788.486175:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718583842023 time 718601914803 state 0x2 ref 0 00000100:00100000:1.0:1789760788.499330:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.499422:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874816 00000100:00100000:2.0:1789760788.499432:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874816 00000100:00100000:2.0:1789760788.499480:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718583842023 time 718601914803 state 0x0 ref 1 00000100:00100000:2.0:1789760788.499495:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718602175219 time 718601914803 state 0x1 ref 1 00000100:00100000:2.0:1789760788.499531:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718602175219 time 718624137992 state 0x0 ref 1 00000100:00100000:2.0:1789760788.499542:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503874816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.499589:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79527936, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ed8 00010000:00010000:2.0:1789760788.499599:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.499614:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.499619:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.499625:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.499638:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.511981:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986723 is committed 00002000:00010000:1.0:1789760788.512142:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79527936 -> 79532032 00002000:00010000:1.0:1789760788.512147:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155352 -> 155360 00010000:00010000:1.0:1789760788.512161:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.512175:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.512186:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.512194:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.512199:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.512209:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.512229:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.512253:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1af1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79527936->79532031] (req 79527936->79532031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.512349:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503874816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12804us (13024us total) trans 4294986723 rc 0/0 00000100:00100000:1.0:1789760788.512360:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19427 00000100:00100000:1.0:1789760788.512370:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718602175219 time 718624137992 state 0x2 ref 0 00000100:00100000:0.0:1789760788.515972:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.516129:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503874944 00000100:00100000:2.0:1789760788.516139:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503874944 00000100:00100000:2.0:1789760788.516196:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718602175219 time 718624137992 state 0x0 ref 1 00000100:00100000:2.0:1789760788.516212:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718624398408 time 718624137992 state 0x1 ref 1 00000100:00100000:2.0:1789760788.516255:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718624398408 time 718640860441 state 0x0 ref 1 00000100:00100000:2.0:1789760788.516268:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503874944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.516317:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79532032, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ee0 00010000:00010000:2.0:1789760788.516327:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.516348:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.516353:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.516361:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.516375:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.523073:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986724 is committed 00002000:00010000:0.0:1789760788.523323:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79532032 -> 79536128 00002000:00010000:0.0:1789760788.523329:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155360 -> 155368 00010000:00010000:0.0:1789760788.523342:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.523355:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.523364:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.523372:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.523376:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.523445:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.523466:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.523474:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1af8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79532032->79536127] (req 79532032->79536127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.523578:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503874944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7307us (7622us total) trans 4294986724 rc 0/0 00000100:00100000:0.0:1789760788.523588:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19428 00000100:00100000:0.0:1789760788.523598:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718624398408 time 718640860441 state 0x2 ref 0 00000100:00100000:2.0:1789760788.527066:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.527144:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875072 00000100:00100000:0.0:1789760788.527154:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875072 00000100:00100000:0.0:1789760788.527219:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718624398408 time 718640860441 state 0x0 ref 1 00000100:00100000:0.0:1789760788.527233:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718641120857 time 718640860441 state 0x1 ref 1 00000100:00100000:0.0:1789760788.527272:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718641120857 time 718651878293 state 0x0 ref 1 00000100:00100000:0.0:1789760788.527285:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503875072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.527332:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79536128, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ee8 00010000:00010000:0.0:1789760788.527342:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527357:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.527361:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.527367:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527380:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.534482:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986725 is committed 00002000:00010000:0.0:1789760788.534674:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79536128 -> 79540224 00002000:00010000:0.0:1789760788.534679:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155368 -> 155376 00010000:00010000:0.0:1789760788.534692:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.534704:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.534713:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.534720:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.534725:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.534734:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.534758:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.534767:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1aff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79536128->79540223] (req 79536128->79540223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.534862:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503875072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7575us (7802us total) trans 4294986725 rc 0/0 00000100:00100000:0.0:1789760788.534872:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19429 00000100:00100000:0.0:1789760788.534882:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718641120857 time 718651878293 state 0x2 ref 0 00000100:00100000:1.0:1789760788.537800:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.537891:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875200 00000100:00100000:2.0:1789760788.537900:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875200 00000100:00100000:2.0:1789760788.537954:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718641120857 time 718651878293 state 0x0 ref 1 00000100:00100000:2.0:1789760788.537969:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718652138709 time 718651878293 state 0x1 ref 1 00000100:00100000:2.0:1789760788.537998:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718652138709 time 718662604738 state 0x0 ref 1 00000100:00100000:2.0:1789760788.538010:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503875200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.538055:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79540224, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ef0 00010000:00010000:2.0:1789760788.538064:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.538077:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.538082:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.538088:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.538101:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.543387:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986726 is committed 00002000:00010000:2.0:1789760788.543837:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79540224 -> 79544320 00002000:00010000:2.0:1789760788.543842:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155376 -> 155384 00010000:00010000:2.0:1789760788.543855:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.543867:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.543875:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.543881:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.543886:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.543894:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.543912:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.543924:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79540224->79544319] (req 79540224->79544319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.544064:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503875200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6052us (6270us total) trans 4294986726 rc 0/0 00000100:00100000:2.0:1789760788.544075:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19430 00000100:00100000:2.0:1789760788.544086:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718652138709 time 718662604738 state 0x2 ref 0 00000100:00100000:1.0:1789760788.554492:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.554594:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875328 00000100:00100000:2.0:1789760788.554603:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875328 00000100:00100000:2.0:1789760788.554659:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718652138709 time 718662604738 state 0x0 ref 1 00000100:00100000:2.0:1789760788.554673:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718662865154 time 718662604738 state 0x1 ref 1 00000100:00100000:2.0:1789760788.554694:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718662865154 time 718679300950 state 0x0 ref 1 00000100:00100000:2.0:1789760788.554706:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503875328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.554752:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79544320, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25ef8 00010000:00010000:2.0:1789760788.554761:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.554777:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.554781:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.554787:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.554807:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.561692:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986727 is committed 00002000:00010000:2.0:1789760788.562026:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79544320 -> 79548416 00002000:00010000:2.0:1789760788.562040:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155384 -> 155392 00010000:00010000:2.0:1789760788.562054:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.562067:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.562075:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.562082:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.562086:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.562096:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.562116:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.562124:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1b0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79544320->79548415] (req 79544320->79548415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.562249:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503875328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7539us (7760us total) trans 4294986727 rc 0/0 00000100:00100000:2.0:1789760788.562260:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19431 00000100:00100000:2.0:1789760788.562270:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718662865154 time 718679300950 state 0x2 ref 0 00000100:00100000:1.0:1789760788.567116:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.567308:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875456 00000100:00100000:0.0:1789760788.567318:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875456 00000100:00100000:0.0:1789760788.567428:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718662865154 time 718679300950 state 0x0 ref 1 00000100:00100000:0.0:1789760788.567443:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718679561366 time 718679300950 state 0x1 ref 1 00000100:00100000:0.0:1789760788.567467:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718679561366 time 718692074324 state 0x0 ref 1 00000100:00100000:0.0:1789760788.567480:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503875456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.567536:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79548416, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f00 00010000:00010000:0.0:1789760788.567545:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.567571:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.567698:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.567705:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.567720:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.572336:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986728 is committed 00002000:00010000:0.0:1789760788.572838:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79548416 -> 79552512 00002000:00010000:0.0:1789760788.572843:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155392 -> 155400 00010000:00010000:0.0:1789760788.572857:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572870:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572878:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572885:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.572890:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572898:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572923:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572931:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1b14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79548416->79552511] (req 79548416->79552511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.573024:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503875456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5541us (5913us total) trans 4294986728 rc 0/0 00000100:00100000:0.0:1789760788.573034:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19432 00000100:00100000:0.0:1789760788.573043:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718679561366 time 718692074324 state 0x2 ref 0 00000100:00100000:1.0:1789760788.578873:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.582952:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875584 00000100:00100000:0.0:1789760788.582968:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875584 00000100:00100000:1.0:1789760788.583818:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718679561366 time 718692074324 state 0x0 ref 1 00000100:00100000:1.0:1789760788.583835:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718692334740 time 718692074324 state 0x1 ref 1 00000100:00100000:1.0:1789760788.583943:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718692334740 time 718708548609 state 0x0 ref 1 00000100:00100000:1.0:1789760788.583957:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503875584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.584000:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79552512, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f08 00010000:00010000:1.0:1789760788.584009:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.584024:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.584028:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.584034:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.584047:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.592618:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986729 is committed 00002000:00010000:1.0:1789760788.592776:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79552512 -> 79556608 00002000:00010000:1.0:1789760788.592781:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155400 -> 155408 00010000:00010000:1.0:1789760788.592793:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.592805:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.592814:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.592821:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.592826:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.592834:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.592854:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.592872:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1b1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79552512->79556607] (req 79552512->79556607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.592960:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503875584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9002us (14092us total) trans 4294986729 rc 0/0 00000100:00100000:1.0:1789760788.592970:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19433 00000100:00100000:1.0:1789760788.592980:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718692334740 time 718708548609 state 0x2 ref 0 00000100:00100000:1.0:1789760788.598531:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.598595:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875712 00000100:00100000:2.0:1789760788.598607:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875712 00000100:00100000:2.0:1789760788.598661:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718692334740 time 718708548609 state 0x0 ref 1 00000100:00100000:2.0:1789760788.598676:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718708809025 time 718708548609 state 0x1 ref 1 00000100:00100000:2.0:1789760788.598707:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718708809025 time 718723313985 state 0x0 ref 1 00000100:00100000:2.0:1789760788.598719:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503875712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.598770:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79556608, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f10 00010000:00010000:2.0:1789760788.598779:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.598796:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.598800:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.598806:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.598819:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.604185:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986730 is committed 00002000:00010000:2.0:1789760788.604287:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79556608 -> 79560704 00002000:00010000:2.0:1789760788.604292:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155408 -> 155416 00010000:00010000:2.0:1789760788.604304:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.604316:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.604324:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.604330:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.604334:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.604343:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.604362:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.604368:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79556608->79560703] (req 79556608->79560703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.604700:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503875712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5979us (6183us total) trans 4294986730 rc 0/0 00000100:00100000:2.0:1789760788.604711:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19434 00000100:00100000:2.0:1789760788.604721:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718708809025 time 718723313985 state 0x2 ref 0 00000100:00100000:2.0:1789760788.607607:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.607682:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875840 00000100:00100000:1.0:1789760788.607693:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875840 00000100:00100000:1.0:1789760788.607749:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718708809025 time 718723313985 state 0x0 ref 1 00000100:00100000:1.0:1789760788.607764:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718723574401 time 718723313985 state 0x1 ref 1 00000100:00100000:1.0:1789760788.607795:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718723574401 time 718732401694 state 0x0 ref 1 00000100:00100000:1.0:1789760788.607809:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503875840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.607855:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79560704, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f18 00010000:00010000:1.0:1789760788.607931:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.607950:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.607955:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.607962:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.607976:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.617271:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986731 is committed 00002000:00010000:1.0:1789760788.617439:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79560704 -> 79564800 00002000:00010000:1.0:1789760788.617444:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155416 -> 155424 00010000:00010000:1.0:1789760788.617457:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.617469:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.617476:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.617483:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.617488:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.617496:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.617531:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.617539:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1b29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79560704->79564799] (req 79560704->79564799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.617632:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503875840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9820us (10030us total) trans 4294986731 rc 0/0 00000100:00100000:1.0:1789760788.617642:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19435 00000100:00100000:1.0:1789760788.617652:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718723574401 time 718732401694 state 0x2 ref 0 00000100:00100000:1.0:1789760788.625971:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.626151:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503875968 00000100:00100000:0.0:1789760788.626161:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503875968 00000100:00100000:0.0:1789760788.626218:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718723574401 time 718732401694 state 0x0 ref 1 00000100:00100000:0.0:1789760788.626234:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718732662110 time 718732401694 state 0x1 ref 1 00000100:00100000:0.0:1789760788.626276:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718732662110 time 718750881795 state 0x0 ref 1 00000100:00100000:0.0:1789760788.626288:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503875968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.626331:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79564800, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f20 00010000:00010000:0.0:1789760788.626341:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.626373:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.626378:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.626384:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.626453:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.631695:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986732 is committed 00002000:00010000:0.0:1789760788.631813:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79564800 -> 79568896 00002000:00010000:0.0:1789760788.631818:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155424 -> 155432 00010000:00010000:0.0:1789760788.631831:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.631842:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.631850:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.631857:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.631930:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.631940:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.631964:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.631971:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1b30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79564800->79568895] (req 79564800->79568895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.636183:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503875968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9883us (10207us total) trans 4294986732 rc 0/0 00000100:00100000:0.0:1789760788.636198:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19436 00000100:00100000:0.0:1789760788.636207:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718732662110 time 718750881795 state 0x2 ref 0 00000100:00100000:0.0:1789760788.641895:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.641981:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876096 00000100:00100000:2.0:1789760788.641991:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876096 00000100:00100000:2.0:1789760788.642044:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718732662110 time 718750881795 state 0x0 ref 1 00000100:00100000:2.0:1789760788.642058:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718751142211 time 718750881795 state 0x1 ref 1 00000100:00100000:2.0:1789760788.642091:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718751142211 time 718766698421 state 0x0 ref 1 00000100:00100000:2.0:1789760788.642103:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.642150:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79568896, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f28 00010000:00010000:2.0:1789760788.642159:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.642175:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.642180:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.642186:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.642200:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.658698:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986733 is committed 00002000:00010000:2.0:1789760788.658819:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79568896 -> 79572992 00002000:00010000:2.0:1789760788.658825:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155432 -> 155440 00010000:00010000:2.0:1789760788.658838:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.658850:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.658859:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.658877:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.658882:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.658892:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.658911:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.658918:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79568896->79572991] (req 79568896->79572991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.659009:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16903us (17119us total) trans 4294986733 rc 0/0 00000100:00100000:2.0:1789760788.659019:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19437 00000100:00100000:2.0:1789760788.659029:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718751142211 time 718766698421 state 0x2 ref 0 00000100:00100000:2.0:1789760788.663263:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.664220:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876224 00000100:00100000:1.0:1789760788.664231:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876224 00000100:00100000:1.0:1789760788.664552:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718751142211 time 718766698421 state 0x0 ref 1 00000100:00100000:1.0:1789760788.664580:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718766958837 time 718766698421 state 0x1 ref 1 00000100:00100000:1.0:1789760788.664649:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718766958837 time 718789254452 state 0x0 ref 1 00000100:00100000:1.0:1789760788.664663:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.664710:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79572992, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f30 00010000:00010000:1.0:1789760788.664719:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.664741:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.664745:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.664751:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.664767:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.673826:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986734 is committed 00002000:00010000:3.0:1789760788.674325:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79572992 -> 79577088 00002000:00010000:3.0:1789760788.674331:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155440 -> 155448 00010000:00010000:3.0:1789760788.674423:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.674505:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.674522:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.674531:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.674535:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.674544:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.674563:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.674571:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1b3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79572992->79577087] (req 79572992->79577087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.674704:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10038us (11446us total) trans 4294986734 rc 0/0 00000100:00100000:3.0:1789760788.674717:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19438 00000100:00100000:3.0:1789760788.674728:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718766958837 time 718789254452 state 0x2 ref 0 00000100:00100000:1.0:1789760788.686010:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.686250:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876352 00000100:00100000:2.0:1789760788.686260:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876352 00000100:00100000:2.0:1789760788.686318:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718766958837 time 718789254452 state 0x0 ref 1 00000100:00100000:2.0:1789760788.686332:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718789514868 time 718789254452 state 0x1 ref 1 00000100:00100000:2.0:1789760788.686358:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718789514868 time 718810964897 state 0x0 ref 1 00000100:00100000:2.0:1789760788.686373:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.686612:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79577088, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f38 00010000:00010000:2.0:1789760788.686622:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686637:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.686641:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.686647:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686662:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.700257:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986735 is committed 00002000:00010000:2.0:1789760788.700613:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79577088 -> 79581184 00002000:00010000:2.0:1789760788.700619:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155448 -> 155456 00010000:00010000:2.0:1789760788.700633:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.700650:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.700659:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.700667:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.700672:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.700681:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.700754:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.700763:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1b45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79577088->79581183] (req 79577088->79581183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.700863:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14487us (14857us total) trans 4294986735 rc 0/0 00000100:00100000:2.0:1789760788.700874:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19439 00000100:00100000:2.0:1789760788.700883:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718789514868 time 718810964897 state 0x2 ref 0 00000100:00100000:0.0:1789760788.705896:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.706656:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876480 00000100:00100000:2.0:1789760788.706667:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876480 00000100:00100000:2.0:1789760788.706734:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718789514868 time 718810964897 state 0x0 ref 1 00000100:00100000:2.0:1789760788.706749:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718811225313 time 718810964897 state 0x1 ref 1 00000100:00100000:2.0:1789760788.706779:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718811225313 time 718831385400 state 0x0 ref 1 00000100:00100000:2.0:1789760788.706792:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503876480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.706838:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79581184, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f40 00010000:00010000:2.0:1789760788.706847:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.706862:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.706866:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.706873:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.706887:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.713333:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986736 is committed 00002000:00010000:2.0:1789760788.713769:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79581184 -> 79585280 00002000:00010000:2.0:1789760788.713775:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155456 -> 155464 00010000:00010000:2.0:1789760788.713787:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.713799:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.713814:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.713821:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.713827:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.713836:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.713855:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.713864:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1b4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79581184->79585279] (req 79581184->79585279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.713992:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503876480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7197us (8101us total) trans 4294986736 rc 0/0 00000100:00100000:2.0:1789760788.714003:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19440 00000100:00100000:2.0:1789760788.714013:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718811225313 time 718831385400 state 0x2 ref 0 00000100:00100000:1.0:1789760788.723015:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.723114:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876608 00000100:00100000:3.0:1789760788.723125:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876608 00000100:00100000:3.0:1789760788.723181:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718811225313 time 718831385400 state 0x0 ref 1 00000100:00100000:3.0:1789760788.723195:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718831645816 time 718831385400 state 0x1 ref 1 00000100:00100000:3.0:1789760788.723226:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718831645816 time 718847833050 state 0x0 ref 1 00000100:00100000:3.0:1789760788.723249:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.723297:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79585280, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f48 00010000:00010000:3.0:1789760788.723306:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.723321:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.723325:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.723331:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.723345:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.732381:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986737 is committed 00002000:00010000:3.0:1789760788.732671:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79585280 -> 79589376 00002000:00010000:3.0:1789760788.732676:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155464 -> 155472 00010000:00010000:3.0:1789760788.732689:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.732702:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.732712:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.732719:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.732724:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.732733:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.732754:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.732762:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79585280->79589375] (req 79585280->79589375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.732905:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9664us (9895us total) trans 4294986737 rc 0/0 00000100:00100000:3.0:1789760788.732915:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19441 00000100:00100000:3.0:1789760788.732925:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718831645816 time 718847833050 state 0x2 ref 0 00000100:00100000:1.0:1789760788.737218:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.737340:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876736 00000100:00100000:3.0:1789760788.737350:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876736 00000100:00100000:3.0:1789760788.737431:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718831645816 time 718847833050 state 0x0 ref 1 00000100:00100000:3.0:1789760788.737446:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718848093466 time 718847833050 state 0x1 ref 1 00000100:00100000:3.0:1789760788.737470:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718848093466 time 718862077368 state 0x0 ref 1 00000100:00100000:3.0:1789760788.737482:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.737535:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79589376, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f50 00010000:00010000:3.0:1789760788.737543:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.737564:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.737568:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.737577:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.737591:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.744372:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986738 is committed 00002000:00010000:3.0:1789760788.744524:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79589376 -> 79593472 00002000:00010000:3.0:1789760788.744530:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155472 -> 155480 00010000:00010000:3.0:1789760788.744544:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.744565:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.744574:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.744582:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.744587:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.744596:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.744616:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.744623:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79589376->79593471] (req 79589376->79593471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.744718:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7232us (7503us total) trans 4294986738 rc 0/0 00000100:00100000:3.0:1789760788.744729:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19442 00000100:00100000:3.0:1789760788.744739:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718848093466 time 718862077368 state 0x2 ref 0 00000100:00100000:0.0:1789760788.748088:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.748191:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876864 00000100:00100000:1.0:1789760788.748202:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876864 00000100:00100000:1.0:1789760788.748268:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718848093466 time 718862077368 state 0x0 ref 1 00000100:00100000:1.0:1789760788.748284:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718862337784 time 718862077368 state 0x1 ref 1 00000100:00100000:1.0:1789760788.748309:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718862337784 time 718872916430 state 0x0 ref 1 00000100:00100000:1.0:1789760788.748322:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.748369:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79593472, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f58 00010000:00010000:1.0:1789760788.748377:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.748435:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760788.748439:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.748446:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.748459:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.755339:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986739 is committed 00002000:00010000:1.0:1789760788.755450:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79593472 -> 79597568 00002000:00010000:1.0:1789760788.755455:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155480 -> 155488 00010000:00010000:1.0:1789760788.755467:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.755479:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.755486:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.755493:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760788.755498:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.755506:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.755542:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.755549:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1b61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79593472->79597567] (req 79593472->79597567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.755647:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503876864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7323us (7564us total) trans 4294986739 rc 0/0 00000100:00100000:1.0:1789760788.755656:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19443 00000100:00100000:1.0:1789760788.755665:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718862337784 time 718872916430 state 0x2 ref 0 00000100:00100000:1.0:1789760788.759349:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.759783:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503876992 00000100:00100000:0.0:1789760788.759794:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503876992 00000100:00100000:0.0:1789760788.759848:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718862337784 time 718872916430 state 0x0 ref 1 00000100:00100000:0.0:1789760788.759862:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718873176846 time 718872916430 state 0x1 ref 1 00000100:00100000:0.0:1789760788.759892:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718873176846 time 718884498485 state 0x0 ref 1 00000100:00100000:0.0:1789760788.759903:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503876992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.759947:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79597568, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f60 00010000:00010000:0.0:1789760788.759956:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.759970:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.759975:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.759980:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.759994:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.766633:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986740 is committed 00002000:00010000:2.0:1789760788.766765:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79597568 -> 79601664 00002000:00010000:2.0:1789760788.766770:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155488 -> 155496 00010000:00010000:2.0:1789760788.766784:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.766801:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.766810:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.766817:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.766821:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.766830:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.766847:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.766854:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1b68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79597568->79601663] (req 79597568->79601663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.766932:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503876992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7026us (7589us total) trans 4294986740 rc 0/0 00000100:00100000:2.0:1789760788.766943:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19444 00000100:00100000:2.0:1789760788.766952:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718873176846 time 718884498485 state 0x2 ref 0 00000100:00100000:2.0:1789760788.776676:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.776859:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503877120 00000100:00100000:3.0:1789760788.776869:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503877120 00000100:00100000:3.0:1789760788.776923:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718873176846 time 718884498485 state 0x0 ref 1 00000100:00100000:3.0:1789760788.776938:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718884758901 time 718884498485 state 0x1 ref 1 00000100:00100000:3.0:1789760788.776969:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718884758901 time 718901575727 state 0x0 ref 1 00000100:00100000:3.0:1789760788.776980:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503877120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.777023:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79601664, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f68 00010000:00010000:3.0:1789760788.777032:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.777047:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.777052:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.777058:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.777070:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.781446:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986741 is committed 00002000:00010000:3.0:1789760788.781627:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79601664 -> 79605760 00002000:00010000:3.0:1789760788.781633:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155496 -> 155504 00010000:00010000:3.0:1789760788.781645:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.781658:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.781668:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.781675:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.781679:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.781688:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.781707:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.781715:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79601664->79605759] (req 79601664->79605759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.781804:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503877120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4821us (5134us total) trans 4294986741 rc 0/0 00000100:00100000:3.0:1789760788.781814:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19445 00000100:00100000:3.0:1789760788.781825:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718884758901 time 718901575727 state 0x2 ref 0 00000100:00100000:0.0:1789760788.785079:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.785203:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503877248 00000100:00100000:3.0:1789760788.785213:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503877248 00000100:00100000:3.0:1789760788.785279:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718884758901 time 718901575727 state 0x0 ref 1 00000100:00100000:3.0:1789760788.785294:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718901836143 time 718901575727 state 0x1 ref 1 00000100:00100000:3.0:1789760788.785324:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718901836143 time 718909930145 state 0x0 ref 1 00000100:00100000:3.0:1789760788.785336:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc10450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.785381:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79605760, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x25f70 00010000:00010000:3.0:1789760788.785422:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785437:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.785442:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.785448:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785461:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.789924:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986742 is committed 00002000:00010000:3.0:1789760788.790035:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79605760 -> 79609856 00002000:00010000:3.0:1789760788.790040:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:3.0:1789760788.790043:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00002000:00010000:3.0:1789760788.790044:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155504 -> 155512 00010000:00010000:3.0:1789760788.790056:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.790073:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.790081:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.790088:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.790092:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.790102:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.790121:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.790129:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79605760->79609855] (req 79605760->79609855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.793217:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc10450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4893us (5158us total) trans 4294986742 rc 0/0 00000100:00100000:3.0:1789760788.793248:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19446 00000100:00100000:3.0:1789760788.793259:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718901836143 time 718909930145 state 0x2 ref 0 00000100:00100000:1.0:1789760788.794090:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.794169:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503877376 00000100:00100000:3.0:1789760788.794179:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503877376 00000100:00100000:3.0:1789760788.794236:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718901836143 time 718909930145 state 0x0 ref 1 00000100:00100000:3.0:1789760788.794258:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718910190561 time 718909930145 state 0x1 ref 1 00000100:00100000:3.0:1789760788.794285:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718910190561 time 718918891692 state 0x0 ref 1 00000100:00100000:3.0:1789760788.794296:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503877376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.794344:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79609856, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25f78 00010000:00010000:3.0:1789760788.794353:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.794368:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.794372:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.794378:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.794631:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.807686:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986743 is committed 00002000:00010000:2.0:1789760788.807809:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79609856 -> 79613952 00002000:00010000:2.0:1789760788.807814:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155512 -> 155520 00010000:00010000:2.0:1789760788.807829:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.807841:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.807849:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.807856:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.807860:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.807869:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.807888:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.807896:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1b7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79609856->79613951] (req 79609856->79613951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.807985:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503877376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13685us (13894us total) trans 4294986743 rc 0/0 00000100:00100000:2.0:1789760788.807995:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19447 00000100:00100000:2.0:1789760788.808005:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718910190561 time 718918891692 state 0x2 ref 0 00000100:00100000:2.0:1789760788.819357:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.819486:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503877504 00000100:00100000:0.0:1789760788.819495:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503877504 00000100:00100000:0.0:1789760788.819563:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718910190561 time 718918891692 state 0x0 ref 1 00000100:00100000:0.0:1789760788.819579:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718919152108 time 718918891692 state 0x1 ref 1 00000100:00100000:0.0:1789760788.819598:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718919152108 time 718944205513 state 0x0 ref 1 00000100:00100000:0.0:1789760788.819610:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.819658:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79613952, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25f80 00010000:00010000:0.0:1789760788.819667:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819687:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760788.819692:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.819698:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819712:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.825708:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986744 is committed 00002000:00010000:0.0:1789760788.828812:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79613952 -> 79618048 00002000:00010000:0.0:1789760788.828822:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155520 -> 155528 00010000:00010000:0.0:1789760788.828836:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.828850:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.828859:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.828867:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.828872:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.828882:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.828908:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.828917:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1b84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79613952->79618047] (req 79613952->79618047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.829020:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9406us (9669us total) trans 4294986744 rc 0/0 00000100:00100000:0.0:1789760788.829031:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19448 00000100:00100000:0.0:1789760788.829040:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718919152108 time 718944205513 state 0x2 ref 0 00000100:00100000:0.0:1789760788.832230:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.832325:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503877632 00000100:00100000:2.0:1789760788.832334:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503877632 00000100:00100000:2.0:1789760788.832413:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718919152108 time 718944205513 state 0x0 ref 1 00000100:00100000:2.0:1789760788.832430:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718944465929 time 718944205513 state 0x1 ref 1 00000100:00100000:2.0:1789760788.832461:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718944465929 time 718957066853 state 0x0 ref 1 00000100:00100000:2.0:1789760788.832472:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.832531:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79618048, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25f88 00010000:00010000:2.0:1789760788.832541:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.832563:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.832567:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.832574:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.832588:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.837654:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986745 is committed 00002000:00010000:2.0:1789760788.839964:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79618048 -> 79622144 00002000:00010000:2.0:1789760788.839975:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155528 -> 155536 00010000:00010000:2.0:1789760788.839988:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840000:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840008:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840015:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.840019:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840027:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840045:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840052:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79618048->79622143] (req 79618048->79622143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.840152:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7676us (7926us total) trans 4294986745 rc 0/0 00000100:00100000:2.0:1789760788.840162:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19449 00000100:00100000:2.0:1789760788.840172:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718944465929 time 718957066853 state 0x2 ref 0 00000100:00100000:1.0:1789760788.845861:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.846182:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503877760 00000100:00100000:3.0:1789760788.846191:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503877760 00000100:00100000:3.0:1789760788.846369:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718944465929 time 718957066853 state 0x0 ref 1 00000100:00100000:3.0:1789760788.846384:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718957327269 time 718957066853 state 0x1 ref 1 00000100:00100000:3.0:1789760788.846478:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718957327269 time 718971084521 state 0x0 ref 1 00000100:00100000:3.0:1789760788.846494:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503877760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.846549:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79622144, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25f90 00010000:00010000:3.0:1789760788.846670:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.846685:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760788.846689:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.846695:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.846710:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.852106:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986746 is committed 00002000:00010000:3.0:1789760788.852531:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79622144 -> 79626240 00002000:00010000:3.0:1789760788.852536:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155536 -> 155544 00010000:00010000:3.0:1789760788.852549:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.852568:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.852577:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.852584:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.852589:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.852598:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.852619:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.852626:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1b92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79622144->79626239] (req 79622144->79626239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.852729:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503877760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6232us (6873us total) trans 4294986746 rc 0/0 00000100:00100000:3.0:1789760788.852740:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19450 00000100:00100000:3.0:1789760788.852750:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718957327269 time 718971084521 state 0x2 ref 0 00000100:00100000:1.0:1789760788.856609:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.857067:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503877888 00000100:00100000:2.0:1789760788.857079:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503877888 00000100:00100000:2.0:1789760788.857132:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718957327269 time 718971084521 state 0x0 ref 1 00000100:00100000:2.0:1789760788.857146:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718971344937 time 718971084521 state 0x1 ref 1 00000100:00100000:2.0:1789760788.857183:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718971344937 time 718981789352 state 0x0 ref 1 00000100:00100000:2.0:1789760788.857195:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.857262:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79626240, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25f98 00010000:00010000:2.0:1789760788.857271:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.857285:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.857290:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.857297:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.857311:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.862488:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986747 is committed 00002000:00010000:2.0:1789760788.862619:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79626240 -> 79630336 00002000:00010000:2.0:1789760788.862624:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155544 -> 155552 00010000:00010000:2.0:1789760788.862637:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.862649:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.862664:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.862672:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.862677:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.862686:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.862705:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.862713:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1b99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79626240->79630335] (req 79626240->79630335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.862804:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503877888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5606us (6200us total) trans 4294986747 rc 0/0 00000100:00100000:2.0:1789760788.862813:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19451 00000100:00100000:2.0:1789760788.862822:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718971344937 time 718981789352 state 0x2 ref 0 00000100:00100000:0.0:1789760788.865669:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.865888:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878016 00000100:00100000:2.0:1789760788.865898:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878016 00000100:00100000:2.0:1789760788.865945:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718971344937 time 718981789352 state 0x0 ref 1 00000100:00100000:2.0:1789760788.865960:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718982049768 time 718981789352 state 0x1 ref 1 00000100:00100000:2.0:1789760788.865988:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718982049768 time 718990595256 state 0x0 ref 1 00000100:00100000:2.0:1789760788.866000:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.866044:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79630336, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fa0 00010000:00010000:2.0:1789760788.866053:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.866068:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760788.866072:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.866079:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.866092:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.869845:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986748 is committed 00002000:00010000:2.0:1789760788.877139:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79630336 -> 79634432 00002000:00010000:2.0:1789760788.877149:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155552 -> 155560 00010000:00010000:2.0:1789760788.877163:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.877176:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.877185:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.877198:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.877204:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.877212:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.877253:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.877262:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1ba0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79630336->79634431] (req 79630336->79634431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.877362:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11359us (11711us total) trans 4294986748 rc 0/0 00000100:00100000:2.0:1789760788.877371:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19452 00000100:00100000:2.0:1789760788.877382:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718982049768 time 718990595256 state 0x2 ref 0 00000100:00100000:1.0:1789760788.881706:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.881824:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878144 00000100:00100000:0.0:1789760788.881835:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878144 00000100:00100000:0.0:1789760788.881889:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718982049768 time 718990595256 state 0x0 ref 1 00000100:00100000:0.0:1789760788.881904:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718990855672 time 718990595256 state 0x1 ref 1 00000100:00100000:0.0:1789760788.881927:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718990855672 time 719006533986 state 0x0 ref 1 00000100:00100000:0.0:1789760788.881939:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503878144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.881984:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79634432, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fa8 00010000:00010000:0.0:1789760788.881992:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.882012:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760788.882015:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.882021:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.882035:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.885606:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986749 is committed 00002000:00010000:0.0:1789760788.885718:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79634432 -> 79638528 00002000:00010000:0.0:1789760788.885724:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155560 -> 155568 00010000:00010000:0.0:1789760788.885737:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.885750:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.885759:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.885765:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.885770:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.885778:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.885803:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.885811:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1ba7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79634432->79638527] (req 79634432->79638527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.885905:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503878144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3962us (4205us total) trans 4294986749 rc 0/0 00000100:00100000:0.0:1789760788.885915:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19453 00000100:00100000:0.0:1789760788.885924:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718990855672 time 719006533986 state 0x2 ref 0 00000100:00100000:1.0:1789760788.890470:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.890668:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878272 00000100:00100000:3.0:1789760788.890678:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878272 00000100:00100000:3.0:1789760788.890730:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 718990855672 time 719006533986 state 0x0 ref 1 00000100:00100000:3.0:1789760788.890745:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719006794402 time 719006533986 state 0x1 ref 1 00000100:00100000:3.0:1789760788.890776:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719006794402 time 719015382540 state 0x0 ref 1 00000100:00100000:3.0:1789760788.890789:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503878272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.890834:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79638528, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fb0 00010000:00010000:3.0:1789760788.890843:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.890859:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760788.890863:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.890869:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.890884:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.894559:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986750 is committed 00002000:00010000:3.0:1789760788.894667:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79638528 -> 79642624 00002000:00010000:3.0:1789760788.894672:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155568 -> 155576 00010000:00010000:3.0:1789760788.894685:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.894697:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.894705:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.894712:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.894716:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.894725:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.894743:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.894750:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79638528->79642623] (req 79638528->79642623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.894851:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503878272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4060us (4387us total) trans 4294986750 rc 0/0 00000100:00100000:3.0:1789760788.894862:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19454 00000100:00100000:3.0:1789760788.894872:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719006794402 time 719015382540 state 0x2 ref 0 00000100:00100000:1.0:1789760788.900480:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.900604:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878400 00000100:00100000:2.0:1789760788.900614:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878400 00000100:00100000:2.0:1789760788.900668:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719006794402 time 719015382540 state 0x0 ref 1 00000100:00100000:2.0:1789760788.900682:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719015642956 time 719015382540 state 0x1 ref 1 00000100:00100000:2.0:1789760788.900712:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719015642956 time 719025318764 state 0x0 ref 1 00000100:00100000:2.0:1789760788.900727:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.900774:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79642624, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fb8 00010000:00010000:2.0:1789760788.900783:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.900799:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760788.900804:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.900810:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.900822:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.903970:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986751 is committed 00002000:00010000:2.0:1789760788.904082:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79642624 -> 79646720 00002000:00010000:2.0:1789760788.904088:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155576 -> 155584 00010000:00010000:2.0:1789760788.904101:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.904114:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.904124:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.904132:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.904138:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.904148:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.904167:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.904174:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79642624->79646719] (req 79642624->79646719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.904276:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 3546us (3800us total) trans 4294986751 rc 0/0 00000100:00100000:2.0:1789760788.904287:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19455 00000100:00100000:2.0:1789760788.904296:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719015642956 time 719025318764 state 0x2 ref 0 00000100:00100000:1.0:1789760788.911217:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.911320:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878528 00000100:00100000:2.0:1789760788.911331:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878528 00000100:00100000:2.0:1789760788.911385:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719015642956 time 719025318764 state 0x0 ref 1 00000100:00100000:2.0:1789760788.911431:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719025579180 time 719025318764 state 0x1 ref 1 00000100:00100000:2.0:1789760788.911456:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719025579180 time 719036063393 state 0x0 ref 1 00000100:00100000:2.0:1789760788.911468:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.911525:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79646720, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fc0 00010000:00010000:2.0:1789760788.911534:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.911548:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760788.911553:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.911560:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.911573:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.914316:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986752 is committed 00002000:00010000:2.0:1789760788.916157:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79646720 -> 79650816 00002000:00010000:2.0:1789760788.916166:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155584 -> 155592 00010000:00010000:2.0:1789760788.916179:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.916192:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.916200:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.916208:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.916213:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.916221:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.916659:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.916668:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79646720->79650815] (req 79646720->79650815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.916821:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5350us (5609us total) trans 4294986752 rc 0/0 00000100:00100000:2.0:1789760788.916832:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19456 00000100:00100000:2.0:1789760788.916841:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719025579180 time 719036063393 state 0x2 ref 0 00000100:00100000:1.0:1789760788.920801:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.920891:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878656 00000100:00100000:0.0:1789760788.920905:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878656 00000100:00100000:0.0:1789760788.920963:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719025579180 time 719036063393 state 0x0 ref 1 00000100:00100000:0.0:1789760788.920979:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719036323809 time 719036063393 state 0x1 ref 1 00000100:00100000:0.0:1789760788.921012:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719036323809 time 719045618730 state 0x0 ref 1 00000100:00100000:0.0:1789760788.921026:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.921078:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79650816, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fc8 00010000:00010000:0.0:1789760788.921089:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.921109:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760788.921117:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.921124:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.921137:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.926870:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986753 is committed 00002000:00010000:0.0:1789760788.927023:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79650816 -> 79654912 00002000:00010000:0.0:1789760788.927028:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155592 -> 155600 00010000:00010000:0.0:1789760788.927040:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.927051:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.927059:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.927065:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.927069:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.927078:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.927095:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.927105:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79650816->79654911] (req 79650816->79654911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.927190:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503878656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6162us (6393us total) trans 4294986753 rc 0/0 00000100:00100000:0.0:1789760788.927200:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19457 00000100:00100000:0.0:1789760788.927210:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719036323809 time 719045618730 state 0x2 ref 0 00000100:00100000:1.0:1789760788.930851:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.930940:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878784 00000100:00100000:3.0:1789760788.930949:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878784 00000100:00100000:3.0:1789760788.931000:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719036323809 time 719045618730 state 0x0 ref 1 00000100:00100000:3.0:1789760788.931013:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719045879146 time 719045618730 state 0x1 ref 1 00000100:00100000:3.0:1789760788.931045:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719045879146 time 719055651262 state 0x0 ref 1 00000100:00100000:3.0:1789760788.931056:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503878784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.931101:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79654912, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fd0 00010000:00010000:3.0:1789760788.931111:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.931130:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760788.931135:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.931141:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.931154:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.935310:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986754 is committed 00002000:00010000:3.0:1789760788.935650:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79654912 -> 79659008 00002000:00010000:3.0:1789760788.935656:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155600 -> 155608 00010000:00010000:3.0:1789760788.935668:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.935681:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.935689:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.935697:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.935701:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.935711:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.935734:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.935743:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1bca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79654912->79659007] (req 79654912->79659007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.935935:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503878784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4875us (5087us total) trans 4294986754 rc 0/0 00000100:00100000:3.0:1789760788.935947:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19458 00000100:00100000:3.0:1789760788.935961:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719045879146 time 719055651262 state 0x2 ref 0 00000100:00100000:1.0:1789760788.939855:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.939945:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503878912 00000100:00100000:2.0:1789760788.939955:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503878912 00000100:00100000:2.0:1789760788.940011:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719045879146 time 719055651262 state 0x0 ref 1 00000100:00100000:2.0:1789760788.940026:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719055911678 time 719055651262 state 0x1 ref 1 00000100:00100000:2.0:1789760788.940057:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719055911678 time 719064664247 state 0x0 ref 1 00000100:00100000:2.0:1789760788.940069:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503878912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.940113:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79659008, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fd8 00010000:00010000:2.0:1789760788.940122:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.940137:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760788.940142:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.940149:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.940163:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.943839:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986755 is committed 00002000:00010000:2.0:1789760788.943945:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79659008 -> 79663104 00002000:00010000:2.0:1789760788.943950:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155608 -> 155616 00010000:00010000:2.0:1789760788.943963:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.943976:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.943984:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.943992:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.943997:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.944006:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.944032:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.944048:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79659008->79663103] (req 79659008->79663103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.944141:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503878912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4069us (4291us total) trans 4294986755 rc 0/0 00000100:00100000:2.0:1789760788.944152:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19459 00000100:00100000:2.0:1789760788.944162:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719055911678 time 719064664247 state 0x2 ref 0 00000100:00100000:0.0:1789760788.948546:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.948626:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879040 00000100:00100000:2.0:1789760788.948636:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879040 00000100:00100000:2.0:1789760788.948691:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719055911678 time 719064664247 state 0x0 ref 1 00000100:00100000:2.0:1789760788.948705:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719064924663 time 719064664247 state 0x1 ref 1 00000100:00100000:2.0:1789760788.948736:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719064924663 time 719073342864 state 0x0 ref 1 00000100:00100000:2.0:1789760788.948749:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.948796:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79663104, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fe0 00010000:00010000:2.0:1789760788.948804:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.948819:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760788.948823:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.948829:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.948848:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.952779:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986756 is committed 00002000:00010000:2.0:1789760788.952880:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79663104 -> 79667200 00002000:00010000:2.0:1789760788.952886:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155616 -> 155624 00010000:00010000:2.0:1789760788.952898:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.952909:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.952918:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.952925:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760788.952930:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.952938:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.952955:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.952962:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1bd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79663104->79667199] (req 79663104->79667199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.953056:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4304us (4514us total) trans 4294986756 rc 0/0 00000100:00100000:2.0:1789760788.953067:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19460 00000100:00100000:2.0:1789760788.953076:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719064924663 time 719073342864 state 0x2 ref 0 00000100:00100000:1.0:1789760788.956208:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760788.956300:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879168 00000100:00100000:0.0:1789760788.956309:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879168 00000100:00100000:0.0:1789760788.956359:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719064924663 time 719073342864 state 0x0 ref 1 00000100:00100000:0.0:1789760788.956373:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719073603280 time 719073342864 state 0x1 ref 1 00000100:00100000:0.0:1789760788.956424:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719073603280 time 719081031275 state 0x0 ref 1 00000100:00100000:0.0:1789760788.956436:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760788.956476:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79667200, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25fe8 00010000:00010000:0.0:1789760788.956486:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.956501:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760788.956505:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.956511:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.956535:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.963718:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986757 is committed 00002000:00010000:0.0:1789760788.963823:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79667200 -> 79671296 00002000:00010000:0.0:1789760788.963828:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155624 -> 155632 00010000:00010000:0.0:1789760788.963841:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.963853:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.963862:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.963869:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.963873:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.963883:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.963901:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.963908:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1bdf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79667200->79671295] (req 79667200->79671295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.963999:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7559us (7795us total) trans 4294986757 rc 0/0 00000100:00100000:0.0:1789760788.964009:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19461 00000100:00100000:0.0:1789760788.964019:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719073603280 time 719081031275 state 0x2 ref 0 00000100:00100000:1.0:1789760788.966642:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760788.966727:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879296 00000100:00100000:3.0:1789760788.966736:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879296 00000100:00100000:3.0:1789760788.966783:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719073603280 time 719081031275 state 0x0 ref 1 00000100:00100000:3.0:1789760788.966797:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719081291691 time 719081031275 state 0x1 ref 1 00000100:00100000:3.0:1789760788.966826:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719081291691 time 719091432453 state 0x0 ref 1 00000100:00100000:3.0:1789760788.966837:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760788.966878:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79671296, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25ff0 00010000:00010000:3.0:1789760788.966887:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.966903:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760788.966907:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.966913:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.966927:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.971345:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986758 is committed 00002000:00010000:3.0:1789760788.971456:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79671296 -> 79675392 00002000:00010000:3.0:1789760788.971461:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155632 -> 155640 00010000:00010000:3.0:1789760788.971474:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.971487:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.971496:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.971503:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760788.971508:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.971525:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.971543:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.971563:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1be6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79671296->79675391] (req 79671296->79675391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.971670:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4830us (5035us total) trans 4294986758 rc 0/0 00000100:00100000:3.0:1789760788.971681:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19462 00000100:00100000:3.0:1789760788.971690:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719081291691 time 719091432453 state 0x2 ref 0 00000100:00100000:2.0:1789760788.979054:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760788.979145:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879424 00000100:00100000:1.0:1789760788.979156:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879424 00000100:00100000:1.0:1789760788.979220:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719081291691 time 719091432453 state 0x0 ref 1 00000100:00100000:1.0:1789760788.979235:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719091692869 time 719091432453 state 0x1 ref 1 00000100:00100000:1.0:1789760788.979265:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719091692869 time 719103871564 state 0x0 ref 1 00000100:00100000:1.0:1789760788.979276:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760788.979319:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79675392, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x25ff8 00010000:00010000:1.0:1789760788.979328:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.979343:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760788.979348:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.979355:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.979368:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.983475:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986759 is committed 00002000:00010000:0.0:1789760788.985416:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79675392 -> 79679488 00002000:00010000:0.0:1789760788.985425:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155640 -> 155648 00010000:00010000:0.0:1789760788.985438:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.985451:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.985459:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.985466:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760788.985471:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.985480:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.985498:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.985511:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1bed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79675392->79679487] (req 79675392->79679487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.985729:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6449us (6679us total) trans 4294986759 rc 0/0 00000100:00100000:0.0:1789760788.985740:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19463 00000100:00100000:0.0:1789760788.985750:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719091692869 time 719103871564 state 0x2 ref 0 00000100:00100000:1.0:1789760788.988500:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760788.988630:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879552 00000100:00100000:2.0:1789760788.988639:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879552 00000100:00100000:2.0:1789760788.988699:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719091692869 time 719103871564 state 0x0 ref 1 00000100:00100000:2.0:1789760788.988713:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719104131980 time 719103871564 state 0x1 ref 1 00000100:00100000:2.0:1789760788.988741:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719104131980 time 719113348400 state 0x0 ref 1 00000100:00100000:2.0:1789760788.988753:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760788.988811:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79679488, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26000 00010000:00010000:2.0:1789760788.988822:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.988836:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760788.988841:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.988847:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.988860:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.002549:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986760 is committed 00002000:00010000:2.0:1789760789.002913:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79679488 -> 79683584 00002000:00010000:2.0:1789760789.002918:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155648 -> 155656 00010000:00010000:2.0:1789760789.002932:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.002945:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.002953:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.002961:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.002965:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.002974:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.002994:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.003002:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1bf4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79679488->79683583] (req 79679488->79683583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.003094:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14338us (14597us total) trans 4294986760 rc 0/0 00000100:00100000:2.0:1789760789.003104:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19464 00000100:00100000:2.0:1789760789.003116:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719104131980 time 719113348400 state 0x2 ref 0 00000100:00100000:1.0:1789760789.009484:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.009637:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879680 00000100:00100000:0.0:1789760789.009647:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879680 00000100:00100000:0.0:1789760789.009704:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719104131980 time 719113348400 state 0x0 ref 1 00000100:00100000:0.0:1789760789.009719:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719113608816 time 719113348400 state 0x1 ref 1 00000100:00100000:0.0:1789760789.009751:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719113608816 time 719134357797 state 0x0 ref 1 00000100:00100000:0.0:1789760789.009762:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.009807:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79683584, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26008 00010000:00010000:0.0:1789760789.009816:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.009830:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.009835:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.009842:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.009859:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.014278:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760789.020104:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986761 is committed 00002000:00010000:0.0:1789760789.020209:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79683584 -> 79687680 00002000:00010000:0.0:1789760789.020215:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155656 -> 155664 00010000:00010000:0.0:1789760789.020228:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.020247:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.020257:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.020264:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.020269:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.020279:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.020297:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.020304:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1bfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79683584->79687679] (req 79683584->79687679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.020426:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10627us (10915us total) trans 4294986761 rc 0/0 00000100:00100000:0.0:1789760789.020437:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19465 00000100:00100000:0.0:1789760789.020455:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719113608816 time 719134357797 state 0x2 ref 0 00000100:00100000:0.0:1789760789.023648:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.023730:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879808 00000100:00100000:3.0:1789760789.023739:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879808 00000100:00100000:3.0:1789760789.023802:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719113608816 time 719134357797 state 0x0 ref 1 00000100:00100000:3.0:1789760789.023817:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719134618213 time 719134357797 state 0x1 ref 1 00000100:00100000:3.0:1789760789.023847:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719134618213 time 719148453883 state 0x0 ref 1 00000100:00100000:3.0:1789760789.023859:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.023904:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79687680, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26010 00010000:00010000:3.0:1789760789.023913:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.023935:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760789.023939:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.023946:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.023960:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.031107:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986762 is committed 00002000:00010000:3.0:1789760789.031253:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79687680 -> 79691776 00002000:00010000:3.0:1789760789.031260:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155664 -> 155672 00010000:00010000:3.0:1789760789.031273:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.031285:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.031294:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.031301:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760789.031306:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.031315:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.031339:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.031345:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79687680->79691775] (req 79687680->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.031511:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503879808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7640us (7860us total) trans 4294986762 rc 0/0 00000100:00100000:3.0:1789760789.031530:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19466 00000100:00100000:3.0:1789760789.031540:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719134618213 time 719148453883 state 0x2 ref 0 00000100:00100000:1.0:1789760789.037864:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.040873:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503879936 00000100:00100000:2.0:1789760789.040889:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503879936 00000100:00100000:2.0:1789760789.040949:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719134618213 time 719148453883 state 0x0 ref 1 00000100:00100000:2.0:1789760789.040964:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719148714299 time 719148453883 state 0x1 ref 1 00000100:00100000:2.0:1789760789.041000:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719148714299 time 719165607152 state 0x0 ref 1 00000100:00100000:2.0:1789760789.041013:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.041061:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79691776, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26018 00010000:00010000:2.0:1789760789.041071:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.041087:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.041091:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.041097:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.041110:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.045079:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986763 is committed 00002000:00010000:2.0:1789760789.045277:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79691776 -> 79695872 00002000:00010000:2.0:1789760789.045283:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155672 -> 155680 00010000:00010000:2.0:1789760789.045296:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.045310:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.045318:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.045326:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.045330:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.045339:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.045358:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.045366:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79691776->79695871] (req 79691776->79695871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.045567:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503879936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4551us (7706us total) trans 4294986763 rc 0/0 00000100:00100000:2.0:1789760789.045578:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19467 00000100:00100000:2.0:1789760789.045587:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719148714299 time 719165607152 state 0x2 ref 0 00000100:00100000:1.0:1789760789.049443:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.050791:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880064 00000100:00100000:2.0:1789760789.050806:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880064 00000100:00100000:2.0:1789760789.050866:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719148714299 time 719165607152 state 0x0 ref 1 00000100:00100000:2.0:1789760789.050881:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719165867568 time 719165607152 state 0x1 ref 1 00000100:00100000:2.0:1789760789.050958:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719165867568 time 719175564596 state 0x0 ref 1 00000100:00100000:2.0:1789760789.050971:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.051018:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79695872, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26020 00010000:00010000:2.0:1789760789.051027:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.051041:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.051045:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.051051:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.051064:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.056815:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986764 is committed 00002000:00010000:2.0:1789760789.056938:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79695872 -> 79699968 00002000:00010000:2.0:1789760789.056944:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155680 -> 155688 00010000:00010000:2.0:1789760789.056956:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.056975:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.056984:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.056992:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.056996:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.057004:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.057023:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.057031:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1c10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79695872->79699967] (req 79695872->79699967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.057125:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6152us (7688us total) trans 4294986764 rc 0/0 00000100:00100000:2.0:1789760789.057135:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19468 00000100:00100000:2.0:1789760789.057145:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719165867568 time 719175564596 state 0x2 ref 0 00000100:00100000:0.0:1789760789.063495:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.064274:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880192 00000100:00100000:1.0:1789760789.064284:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880192 00000100:00100000:1.0:1789760789.064340:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719165867568 time 719175564596 state 0x0 ref 1 00000100:00100000:1.0:1789760789.064355:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719175825012 time 719175564596 state 0x1 ref 1 00000100:00100000:1.0:1789760789.064387:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719175825012 time 719188993782 state 0x0 ref 1 00000100:00100000:1.0:1789760789.064912:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.064961:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79699968, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26028 00010000:00010000:1.0:1789760789.064971:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.064986:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.064991:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.064999:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.065013:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.070465:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986765 is committed 00002000:00010000:1.0:1789760789.073723:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79699968 -> 79704064 00002000:00010000:1.0:1789760789.073732:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155688 -> 155696 00010000:00010000:1.0:1789760789.073747:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.073760:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.073769:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.073777:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.073782:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.073852:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.073872:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.073887:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1c17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79699968->79704063] (req 79699968->79704063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.073989:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9077us (10499us total) trans 4294986765 rc 0/0 00000100:00100000:1.0:1789760789.073999:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19469 00000100:00100000:1.0:1789760789.074009:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719175825012 time 719188993782 state 0x2 ref 0 00000100:00100000:1.0:1789760789.078351:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.078534:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880320 00000100:00100000:0.0:1789760789.078545:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880320 00000100:00100000:0.0:1789760789.078610:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719175825012 time 719188993782 state 0x0 ref 1 00000100:00100000:0.0:1789760789.078624:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719189254198 time 719188993782 state 0x1 ref 1 00000100:00100000:0.0:1789760789.078654:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719189254198 time 719203260966 state 0x0 ref 1 00000100:00100000:0.0:1789760789.078666:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.078714:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79704064, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26030 00010000:00010000:0.0:1789760789.078723:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.078737:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.078741:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.078747:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.078766:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.083042:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986766 is committed 00002000:00010000:0.0:1789760789.083145:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79704064 -> 79708160 00002000:00010000:0.0:1789760789.083150:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155696 -> 155704 00010000:00010000:0.0:1789760789.083162:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.083175:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.083183:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.083190:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.083205:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.083214:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.083235:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.083242:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79704064->79708159] (req 79704064->79708159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.083870:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4832us (5155us total) trans 4294986766 rc 0/0 00000100:00100000:0.0:1789760789.083881:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19470 00000100:00100000:0.0:1789760789.083890:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719189254198 time 719203260966 state 0x2 ref 0 00000100:00100000:2.0:1789760789.092281:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.092482:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880448 00000100:00100000:3.0:1789760789.092494:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880448 00000100:00100000:3.0:1789760789.092847:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719189254198 time 719203260966 state 0x0 ref 1 00000100:00100000:3.0:1789760789.092862:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719203521382 time 719203260966 state 0x1 ref 1 00000100:00100000:3.0:1789760789.092891:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719203521382 time 719217498232 state 0x0 ref 1 00000100:00100000:3.0:1789760789.092904:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503880448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.093066:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79708160, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26038 00010000:00010000:3.0:1789760789.093077:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.093099:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760789.093103:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.093110:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.093124:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.099152:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986767 is committed 00002000:00010000:3.0:1789760789.099363:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79708160 -> 79712256 00002000:00010000:3.0:1789760789.099368:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155704 -> 155712 00010000:00010000:3.0:1789760789.099380:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.099921:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.099931:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.099938:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760789.100001:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.100013:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.100039:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.100047:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1c25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79708160->79712255] (req 79708160->79712255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.103849:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503880448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10935us (11564us total) trans 4294986767 rc 0/0 00000100:00100000:0.0:1789760789.103863:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19471 00000100:00100000:0.0:1789760789.103873:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719203521382 time 719217498232 state 0x2 ref 0 00000100:00100000:2.0:1789760789.110132:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.110321:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880576 00000100:00100000:0.0:1789760789.110331:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880576 00000100:00100000:0.0:1789760789.110385:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719203521382 time 719217498232 state 0x0 ref 1 00000100:00100000:0.0:1789760789.110443:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719217758648 time 719217498232 state 0x1 ref 1 00000100:00100000:0.0:1789760789.110472:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719217758648 time 719235078510 state 0x0 ref 1 00000100:00100000:0.0:1789760789.110484:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.110558:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79712256, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26040 00010000:00010000:0.0:1789760789.110568:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.110584:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.110588:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.110595:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.110608:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.121849:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986768 is committed 00002000:00010000:0.0:1789760789.121989:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79712256 -> 79716352 00002000:00010000:0.0:1789760789.121994:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155712 -> 155720 00010000:00010000:0.0:1789760789.122009:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.122032:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.122041:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.122048:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.122053:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.122061:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.122079:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.122090:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1c2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79712256->79716351] (req 79712256->79716351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.122349:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11860us (12221us total) trans 4294986768 rc 0/0 00000100:00100000:0.0:1789760789.122359:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19472 00000100:00100000:0.0:1789760789.122369:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719217758648 time 719235078510 state 0x2 ref 0 00000100:00100000:1.0:1789760789.125666:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.125773:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880704 00000100:00100000:2.0:1789760789.125791:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880704 00000100:00100000:2.0:1789760789.125844:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719217758648 time 719235078510 state 0x0 ref 1 00000100:00100000:2.0:1789760789.125860:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719235338926 time 719235078510 state 0x1 ref 1 00000100:00100000:2.0:1789760789.125890:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719235338926 time 719250496224 state 0x0 ref 1 00000100:00100000:2.0:1789760789.125903:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.125961:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79716352, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26048 00010000:00010000:2.0:1789760789.125971:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.125987:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.125991:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.125998:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.126011:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.135239:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986769 is committed 00002000:00010000:2.0:1789760789.135444:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79716352 -> 79720448 00002000:00010000:2.0:1789760789.135449:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155720 -> 155728 00010000:00010000:2.0:1789760789.135462:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.135480:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.135489:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.135496:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.135500:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.135510:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.135539:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.135546:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1c33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79716352->79720447] (req 79716352->79720447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.135910:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10004us (10248us total) trans 4294986769 rc 0/0 00000100:00100000:2.0:1789760789.135921:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19473 00000100:00100000:2.0:1789760789.135930:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719235338926 time 719250496224 state 0x2 ref 0 00000100:00100000:2.0:1789760789.140594:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.140792:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880832 00000100:00100000:0.0:1789760789.140802:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880832 00000100:00100000:0.0:1789760789.140856:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719235338926 time 719250496224 state 0x0 ref 1 00000100:00100000:0.0:1789760789.140871:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719250756640 time 719250496224 state 0x1 ref 1 00000100:00100000:0.0:1789760789.140899:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719250756640 time 719265506009 state 0x0 ref 1 00000100:00100000:0.0:1789760789.140912:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.140957:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79720448, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26050 00010000:00010000:0.0:1789760789.140968:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.140983:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.140987:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.140993:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.141011:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.152147:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986770 is committed 00002000:00010000:0.0:1789760789.152362:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79720448 -> 79724544 00002000:00010000:0.0:1789760789.152367:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155728 -> 155736 00010000:00010000:0.0:1789760789.152381:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.152422:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.152432:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.152440:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.152445:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.152454:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.152474:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.152482:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1c3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79720448->79724543] (req 79720448->79724543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.153352:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503880832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12434us (12759us total) trans 4294986770 rc 0/0 00000100:00100000:0.0:1789760789.153364:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19474 00000100:00100000:0.0:1789760789.153373:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719250756640 time 719265506009 state 0x2 ref 0 00000100:00100000:0.0:1789760789.157365:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.157830:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503880960 00000100:00100000:1.0:1789760789.157841:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503880960 00000100:00100000:1.0:1789760789.157898:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719250756640 time 719265506009 state 0x0 ref 1 00000100:00100000:1.0:1789760789.157974:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719265766425 time 719265506009 state 0x1 ref 1 00000100:00100000:1.0:1789760789.158042:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719265766425 time 719282648114 state 0x0 ref 1 00000100:00100000:1.0:1789760789.158056:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503880960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.158104:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79724544, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26058 00010000:00010000:1.0:1789760789.158114:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.158130:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.158134:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.158139:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.158153:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.164797:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986771 is committed 00002000:00010000:1.0:1789760789.164949:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79724544 -> 79728640 00002000:00010000:1.0:1789760789.164954:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155736 -> 155744 00010000:00010000:1.0:1789760789.164967:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.164980:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.164988:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.164996:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.165001:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.165010:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.165032:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.165039:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79724544->79728639] (req 79724544->79728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.165131:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503880960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7073us (7772us total) trans 4294986771 rc 0/0 00000100:00100000:1.0:1789760789.165142:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19475 00000100:00100000:1.0:1789760789.165153:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719265766425 time 719282648114 state 0x2 ref 0 00000100:00100000:1.0:1789760789.171381:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.171685:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881088 00000100:00100000:2.0:1789760789.171695:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881088 00000100:00100000:2.0:1789760789.171750:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719265766425 time 719282648114 state 0x0 ref 1 00000100:00100000:2.0:1789760789.171766:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719282908530 time 719282648114 state 0x1 ref 1 00000100:00100000:2.0:1789760789.171796:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719282908530 time 719296403096 state 0x0 ref 1 00000100:00100000:2.0:1789760789.171809:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503881088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.171866:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79728640, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26060 00010000:00010000:2.0:1789760789.171874:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.171891:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.171895:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.171901:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.171913:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.181931:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986772 is committed 00002000:00010000:2.0:1789760789.182084:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79728640 -> 79732736 00002000:00010000:2.0:1789760789.182089:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155744 -> 155752 00010000:00010000:2.0:1789760789.182102:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.182115:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.182125:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.182133:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.182138:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.182148:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.182168:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.182176:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1c48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79728640->79732735] (req 79728640->79732735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.182485:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503881088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10671us (11107us total) trans 4294986772 rc 0/0 00000100:00100000:2.0:1789760789.182496:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19476 00000100:00100000:2.0:1789760789.182505:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719282908530 time 719296403096 state 0x2 ref 0 00000100:00100000:2.0:1789760789.191144:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.191684:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881216 00000100:00100000:1.0:1789760789.191695:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881216 00000100:00100000:1.0:1789760789.191750:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719282908530 time 719296403096 state 0x0 ref 1 00000100:00100000:1.0:1789760789.191766:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719296663512 time 719296403096 state 0x1 ref 1 00000100:00100000:1.0:1789760789.191841:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719296663512 time 719316447268 state 0x0 ref 1 00000100:00100000:1.0:1789760789.191855:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503881216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.191898:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79732736, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26068 00010000:00010000:1.0:1789760789.191906:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.191926:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.191931:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.191937:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.191965:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.197063:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986773 is committed 00002000:00010000:1.0:1789760789.200531:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79732736 -> 79736832 00002000:00010000:1.0:1789760789.200541:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155752 -> 155760 00010000:00010000:1.0:1789760789.200555:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.200701:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.200710:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.200718:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.200723:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.200732:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.200759:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.200768:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79732736->79736831] (req 79732736->79736831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.200943:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503881216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9085us (9803us total) trans 4294986773 rc 0/0 00000100:00100000:1.0:1789760789.200955:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19477 00000100:00100000:1.0:1789760789.200964:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719296663512 time 719316447268 state 0x2 ref 0 00000100:00100000:1.0:1789760789.202863:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cce83d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523613056:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760789.202934:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203155:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3638:1876699523613568:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203186:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203213:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523614080:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203236:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203253:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9060700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523614592:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203273:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203293:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1789760789.203299:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:afc717a1-11ba-4ef0-ab10-103aa692ca98:3638:1876699523613184:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203326:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203345:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523613696:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203368:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203385:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc8f73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523614208:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203435:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203462:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613568 00000100:00100000:2.0:1789760789.203468:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613568 00000100:00100000:2.0:1789760789.203526:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 985 00000100:00100000:2.0:1789760789.203536:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3638:x1876699523613568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203596:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9061180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3638:x1876699523613568:12345-0@lo:400:kworker.0 Request processed in 61us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.203606:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 985 00000100:00100000:3.0:1789760789.203607:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3636:1876699523613312:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203644:0:15162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614080 00000100:00100000:3.0:1789760789.203649:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203650:0:15162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614080 00000100:00100000:3.0:1789760789.203677:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3636:1876699523613824:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203690:0:15162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614592 00000100:00100000:2.0:1789760789.203694:0:15162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614592 00000100:00100000:3.0:1789760789.203704:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203721:0:15162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614208 00000100:00100000:3.0:1789760789.203723:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9062a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3636:1876699523614336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.203726:0:15162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614208 00000100:00100000:2.0:1789760789.203751:0:15162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:3.0:1789760789.203761:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.203762:0:15162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523614080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.203844:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523612928:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760789.204017:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613312 00000100:00100000:3.0:1789760789.204024:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613312 00000100:00100000:3.0:1789760789.204056:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613824 00000100:00100000:3.0:1789760789.204061:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613824 00000100:00100000:3.0:1789760789.204078:0:6512:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614336 00000100:00100000:3.0:1789760789.204081:0:6512:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614336 00000100:00100000:3.0:1789760789.204102:0:6512:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 986 00000100:00100000:3.0:1789760789.204111:0:6512:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3636:x1876699523613312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760789.204444:0:8490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:1.0:1789760789.204455:0:8490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523614592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204534:0:15162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523614080:12345-0@lo:400:kworker.0 Request processed in 777us (1297us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.204545:0:15162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1789760789.204570:0:15162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:2.0:1789760789.204577:0:15162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523614208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204608:0:15162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523614208:12345-0@lo:400:kworker.0 Request processed in 33us (1172us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.204623:0:15162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:2.0:1789760789.204646:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613696 00000100:00100000:2.0:1789760789.204651:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613696 00000100:00100000:2.0:1789760789.204679:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1789760789.204685:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523613696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204718:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523613696:12345-0@lo:400:kworker.0 Request processed in 33us (1349us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.204723:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1789760789.204741:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 987 00000100:00100000:2.0:1789760789.204748:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3636:x1876699523613824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204784:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3636:x1876699523613824:12345-0@lo:400:kworker.0 Request processed in 36us (1080us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.204789:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 987 00000100:00100000:2.0:1789760789.204806:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 988 00000100:00100000:2.0:1789760789.204812:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3636:x1876699523614336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204837:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3636:x1876699523614336:12345-0@lo:400:kworker.0 Request processed in 26us (1082us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.204843:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 988 00000100:00080000:2.0:1789760789.204859:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760789.204863:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760789.204867:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760789.204909:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3636:1876699523613824:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204922:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9062a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3636:1876699523614336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204963:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523613696:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204976:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc8f73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523614208:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204985:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3638:1876699523613568:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.204994:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523614080:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.205101:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.205136:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613056 00000100:00100000:2.0:1789760789.205141:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613056 00000100:00100000:2.0:1789760789.205172:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789760789.205178:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523613056:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760789.205199:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760789.205234:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3638:x1876699523613056:12345-0@lo:13:osp-pre-1-1.0 Request processed in 56us (2300us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.205241:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789760789.205259:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613184 00000100:00100000:2.0:1789760789.205262:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613184 00000100:00100000:2.0:1789760789.205285:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789760789.205290:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3638:x1876699523613184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.205321:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3638:x1876699523613184:12345-0@lo:400:kworker.0 Request processed in 31us (1994us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.205327:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00080000:2.0:1789760789.205355:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760789.205370:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:afc717a1-11ba-4ef0-ab10-103aa692ca98:3638:1876699523613184:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.205386:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cce83d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523613056:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760789.206257:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760789.206296:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523613440:0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206328:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760789.206349:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cd24a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3637:1876699523613952:0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206372:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760789.206405:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc9060380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3637:1876699523614464:0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206428:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760789.206469:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613440 00000100:00100000:0.0:1789760789.206474:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613440 00000100:00100000:0.0:1789760789.206509:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:0.0:1789760789.206528:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523613440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206581:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3637:x1876699523613440:12345-0@lo:400:kworker.0 Request processed in 55us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.206588:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:0.0:1789760789.206612:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523613952 00000100:00100000:0.0:1789760789.206625:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523613952 00000100:00100000:0.0:1789760789.206654:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614464 00000100:00100000:0.0:1789760789.206657:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614464 00000100:00100000:0.0:1789760789.206678:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 989 00000100:00100000:0.0:1789760789.206684:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3637:x1876699523613952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206716:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca081500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3637:x1876699523613952:12345-0@lo:400:kworker.0 Request processed in 32us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.206722:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 989 00000100:00100000:0.0:1789760789.206733:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 990 00000100:00100000:0.0:1789760789.206739:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3637:x1876699523614464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206765:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca080380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3637:x1876699523614464:12345-0@lo:400:kworker.0 Request processed in 26us (336us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.206770:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 990 00000100:00080000:0.0:1789760789.206787:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760789.206814:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523613440:0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206824:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3637:1876699523613952:0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206832:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9060380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3637:1876699523614464:0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.206855:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881344 00000100:00100000:0.0:1789760789.206858:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881344 00000100:00100000:0.0:1789760789.206895:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719296663512 time 719316447268 state 0x0 ref 1 00000100:00100000:0.0:1789760789.206907:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719316707684 time 719316447268 state 0x1 ref 1 00000100:00100000:0.0:1789760789.206927:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719316707684 time 719331534209 state 0x0 ref 1 00000100:00100000:0.0:1789760789.206937:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503881344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.206976:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79736832, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26070 00010000:00010000:0.0:1789760789.206985:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.207001:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.207005:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.207012:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.207024:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.207703:0:6512:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3636:x1876699523613312:12345-0@lo:400:kworker.0 Request processed in 3589us (4052us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760789.207713:0:6512:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 986 00000100:00100000:2.0:1789760789.207878:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523612928 00000100:00100000:2.0:1789760789.207884:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523612928 00000100:00100000:2.0:1789760789.207924:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789760789.207931:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3637:x1876699523612928:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760789.207948:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760789.208002:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ddfa00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3637:x1876699523612928:12345-0@lo:13:osp-pre-1-0.0 Request processed in 71us (1747us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.208008:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789760789.208044:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3636:1876699523613312:0@lo:400:kworker.0 00000100:00100000:3.0:1789760789.208253:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cd24a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523612928:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760789.208768:0:8490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523614592:12345-0@lo:400:kworker.0 Request processed in 4314us (5491us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760789.208779:0:8490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1789760789.208840:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc9060700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523614592:0@lo:400:kworker.0 00000001:00080000:3.0:1789760789.211823:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986774 is committed 00002000:00010000:0.0:1789760789.211970:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79736832 -> 79740928 00002000:00010000:0.0:1789760789.211976:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155760 -> 155768 00010000:00010000:0.0:1789760789.211991:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.212005:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.212016:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.212024:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.212028:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.212038:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.212059:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.212066:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79736832->79740927] (req 79736832->79740927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.212163:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503881344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5221us (7057us total) trans 4294986774 rc 0/0 00000100:00100000:0.0:1789760789.212174:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19478 00000100:00100000:0.0:1789760789.212184:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719316707684 time 719331534209 state 0x2 ref 0 00000100:00100000:0.0:1789760789.221616:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.221698:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881472 00000100:00100000:1.0:1789760789.221709:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881472 00000100:00100000:1.0:1789760789.221765:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719316707684 time 719331534209 state 0x0 ref 1 00000100:00100000:1.0:1789760789.221780:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719331794625 time 719331534209 state 0x1 ref 1 00000100:00100000:1.0:1789760789.221830:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719331794625 time 719346434158 state 0x0 ref 1 00000100:00100000:1.0:1789760789.221845:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503881472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.221900:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79740928, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26078 00010000:00010000:1.0:1789760789.221912:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.221929:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.221934:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.221941:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.221956:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.229015:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986775 is committed 00002000:00010000:1.0:1789760789.229693:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79740928 -> 79745024 00002000:00010000:1.0:1789760789.229700:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155768 -> 155776 00010000:00010000:1.0:1789760789.229712:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.229725:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.229734:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.229742:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.229747:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.229756:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.229774:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.229781:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79740928->79745023] (req 79740928->79745023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.229874:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cca082d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503881472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8028us (8263us total) trans 4294986775 rc 0/0 00000100:00100000:1.0:1789760789.229884:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19479 00000100:00100000:1.0:1789760789.229893:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719331794625 time 719346434158 state 0x2 ref 0 00000100:00100000:2.0:1789760789.237012:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.237125:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881600 00000100:00100000:0.0:1789760789.237136:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881600 00000100:00100000:0.0:1789760789.237195:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719331794625 time 719346434158 state 0x0 ref 1 00000100:00100000:0.0:1789760789.237211:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719346694574 time 719346434158 state 0x1 ref 1 00000100:00100000:0.0:1789760789.237244:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719346694574 time 719361850352 state 0x0 ref 1 00000100:00100000:0.0:1789760789.237256:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503881600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.237300:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79745024, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26080 00010000:00010000:0.0:1789760789.237310:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.237333:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.237338:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.237344:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.237358:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.242065:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986776 is committed 00002000:00010000:2.0:1789760789.242695:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79745024 -> 79749120 00002000:00010000:2.0:1789760789.242701:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155776 -> 155784 00010000:00010000:2.0:1789760789.242714:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.242727:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.242736:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.242743:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.242749:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.242758:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.242777:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.242784:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1c64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79745024->79749119] (req 79745024->79749119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.242913:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503881600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5654us (5906us total) trans 4294986776 rc 0/0 00000100:00100000:2.0:1789760789.242924:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19480 00000100:00100000:2.0:1789760789.242938:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719346694574 time 719361850352 state 0x2 ref 0 00000100:00100000:2.0:1789760789.246967:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.248012:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881728 00000100:00100000:1.0:1789760789.248023:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881728 00000100:00100000:1.0:1789760789.248081:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719346694574 time 719361850352 state 0x0 ref 1 00000100:00100000:1.0:1789760789.248097:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719362110768 time 719361850352 state 0x1 ref 1 00000100:00100000:1.0:1789760789.248120:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719362110768 time 719372727319 state 0x0 ref 1 00000100:00100000:1.0:1789760789.248132:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503881728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.248837:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79749120, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26088 00010000:00010000:1.0:1789760789.248850:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.248865:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.248870:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.248877:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.248891:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.253471:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986777 is committed 00002000:00010000:1.0:1789760789.253603:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79749120 -> 79753216 00002000:00010000:1.0:1789760789.253608:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155784 -> 155792 00010000:00010000:1.0:1789760789.253623:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.253635:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.253644:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.253650:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.253655:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.253663:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.253686:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.253694:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79749120->79753215] (req 79749120->79753215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.253785:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503881728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5649us (6823us total) trans 4294986777 rc 0/0 00000100:00100000:1.0:1789760789.253796:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19481 00000100:00100000:1.0:1789760789.253807:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719362110768 time 719372727319 state 0x2 ref 0 00000100:00100000:2.0:1789760789.256255:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.256332:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881856 00000100:00100000:0.0:1789760789.256342:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881856 00000100:00100000:0.0:1789760789.256423:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719362110768 time 719372727319 state 0x0 ref 1 00000100:00100000:0.0:1789760789.256441:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719372987735 time 719372727319 state 0x1 ref 1 00000100:00100000:0.0:1789760789.256471:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719372987735 time 719381077914 state 0x0 ref 1 00000100:00100000:0.0:1789760789.256485:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503881856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.256543:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79753216, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26090 00010000:00010000:0.0:1789760789.256552:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.256568:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.256573:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.256579:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.256606:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.260750:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986778 is committed 00002000:00010000:0.0:1789760789.261090:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79753216 -> 79757312 00002000:00010000:0.0:1789760789.261095:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155792 -> 155800 00010000:00010000:0.0:1789760789.261107:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.261119:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.261127:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.261136:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.261141:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.261149:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.261180:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.261187:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79753216->79757311] (req 79753216->79757311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.261280:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503881856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4793us (5039us total) trans 4294986778 rc 0/0 00000100:00100000:0.0:1789760789.261289:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19482 00000100:00100000:0.0:1789760789.261298:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719372987735 time 719381077914 state 0x2 ref 0 00000100:00100000:2.0:1789760789.264996:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.265076:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503881984 00000100:00100000:0.0:1789760789.265086:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503881984 00000100:00100000:0.0:1789760789.265143:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719372987735 time 719381077914 state 0x0 ref 1 00000100:00100000:0.0:1789760789.265170:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719381338330 time 719381077914 state 0x1 ref 1 00000100:00100000:0.0:1789760789.265203:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719381338330 time 719389809015 state 0x0 ref 1 00000100:00100000:0.0:1789760789.265216:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503881984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.265261:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79757312, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26098 00010000:00010000:0.0:1789760789.265271:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.265287:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.265291:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.265298:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.265311:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.269902:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986779 is committed 00002000:00010000:0.0:1789760789.270021:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79757312 -> 79761408 00002000:00010000:0.0:1789760789.270026:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155800 -> 155808 00010000:00010000:0.0:1789760789.270040:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.270052:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.270061:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.270067:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.270071:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.270080:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.270098:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.270105:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79757312->79761407] (req 79757312->79761407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.270206:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503881984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4989us (5215us total) trans 4294986779 rc 0/0 00000100:00100000:0.0:1789760789.270216:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19483 00000100:00100000:0.0:1789760789.270224:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719381338330 time 719389809015 state 0x2 ref 0 00000100:00100000:2.0:1789760789.272584:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.272739:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503882112 00000100:00100000:0.0:1789760789.272749:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503882112 00000100:00100000:0.0:1789760789.272800:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719381338330 time 719389809015 state 0x0 ref 1 00000100:00100000:0.0:1789760789.272814:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719390069431 time 719389809015 state 0x1 ref 1 00000100:00100000:0.0:1789760789.272845:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719390069431 time 719397452271 state 0x0 ref 1 00000100:00100000:0.0:1789760789.272856:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.272900:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79761408, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260a0 00010000:00010000:0.0:1789760789.272907:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.272921:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.272925:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.272932:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.272944:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.276634:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986780 is committed 00002000:00010000:0.0:1789760789.276781:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79761408 -> 79765504 00002000:00010000:0.0:1789760789.276787:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155808 -> 155816 00010000:00010000:0.0:1789760789.276800:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.276812:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.276821:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.276828:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.276832:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.276840:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.276859:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.276865:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79761408->79765503] (req 79761408->79765503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.276952:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc597fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4092us (4372us total) trans 4294986780 rc 0/0 00000100:00100000:0.0:1789760789.276961:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19484 00000100:00100000:0.0:1789760789.276971:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719390069431 time 719397452271 state 0x2 ref 0 00000100:00100000:2.0:1789760789.281485:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.282855:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503882240 00000100:00100000:1.0:1789760789.282868:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503882240 00000100:00100000:1.0:1789760789.282925:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719390069431 time 719397452271 state 0x0 ref 1 00000100:00100000:1.0:1789760789.282940:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719397712687 time 719397452271 state 0x1 ref 1 00000100:00100000:1.0:1789760789.282973:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719397712687 time 719407579602 state 0x0 ref 1 00000100:00100000:1.0:1789760789.282986:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503882240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.283033:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79765504, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260a8 00010000:00010000:1.0:1789760789.283042:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.283062:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.283066:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.283072:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.283085:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.290081:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986781 is committed 00002000:00010000:1.0:1789760789.290439:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79765504 -> 79769600 00002000:00010000:1.0:1789760789.290444:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155816 -> 155824 00010000:00010000:1.0:1789760789.290457:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.290470:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.290479:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.290486:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.290491:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.290500:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.290537:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.290545:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1c87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79765504->79769599] (req 79765504->79769599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.290868:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503882240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7879us (9387us total) trans 4294986781 rc 0/0 00000100:00100000:1.0:1789760789.290878:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19485 00000100:00100000:1.0:1789760789.290890:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719397712687 time 719407579602 state 0x2 ref 0 00000100:00100000:2.0:1789760789.298148:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.298255:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503882368 00000100:00100000:0.0:1789760789.298266:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503882368 00000100:00100000:0.0:1789760789.298313:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719397712687 time 719407579602 state 0x0 ref 1 00000100:00100000:0.0:1789760789.298329:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719407840018 time 719407579602 state 0x1 ref 1 00000100:00100000:0.0:1789760789.298361:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719407840018 time 719422968155 state 0x0 ref 1 00000100:00100000:0.0:1789760789.298373:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.298452:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79769600, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260b0 00010000:00010000:0.0:1789760789.298460:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.298475:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.298479:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.298484:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.298496:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.305478:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986782 is committed 00002000:00010000:0.0:1789760789.307684:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79769600 -> 79773696 00002000:00010000:0.0:1789760789.307694:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155824 -> 155832 00010000:00010000:0.0:1789760789.307711:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.307827:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.307836:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.307843:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.307848:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.307857:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.307877:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.307884:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f1c8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79769600->79773695] (req 79769600->79773695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.307990:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9613us (9846us total) trans 4294986782 rc 0/0 00000100:00100000:0.0:1789760789.308000:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19486 00000100:00100000:0.0:1789760789.308010:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719407840018 time 719422968155 state 0x2 ref 0 00000100:00100000:2.0:1789760789.313835:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.317175:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503882496 00000100:00100000:0.0:1789760789.317189:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503882496 00000100:00100000:0.0:1789760789.317264:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719407840018 time 719422968155 state 0x0 ref 1 00000100:00100000:0.0:1789760789.317279:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719423228571 time 719422968155 state 0x1 ref 1 00000100:00100000:0.0:1789760789.317299:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719423228571 time 719441906817 state 0x0 ref 1 00000100:00100000:0.0:1789760789.317311:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503882496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.317352:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79773696, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260b8 00010000:00010000:0.0:1789760789.317360:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.317375:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.317379:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.317385:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.317422:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.330098:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986783 is committed 00002000:00010000:2.0:1789760789.330267:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79773696 -> 79777792 00002000:00010000:2.0:1789760789.330272:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155832 -> 155840 00010000:00010000:2.0:1789760789.330295:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.330308:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.330317:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.330323:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.330328:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.330337:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.330357:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.330364:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1c95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79773696->79777791] (req 79773696->79777791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.330495:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503882496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13180us (16664us total) trans 4294986783 rc 0/0 00000100:00100000:2.0:1789760789.330505:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19487 00000100:00100000:2.0:1789760789.330525:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719423228571 time 719441906817 state 0x2 ref 0 00000100:00100000:2.0:1789760789.336898:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.337018:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503882624 00000100:00100000:0.0:1789760789.337028:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503882624 00000100:00100000:0.0:1789760789.337082:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719423228571 time 719441906817 state 0x0 ref 1 00000100:00100000:0.0:1789760789.337098:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719442167233 time 719441906817 state 0x1 ref 1 00000100:00100000:0.0:1789760789.337168:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719442167233 time 719461774107 state 0x0 ref 1 00000100:00100000:0.0:1789760789.337181:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.337229:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79777792, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260c0 00010000:00010000:0.0:1789760789.337546:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.337634:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.337639:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.337645:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.337660:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.345087:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986784 is committed 00002000:00010000:0.0:1789760789.346293:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79777792 -> 79781888 00002000:00010000:0.0:1789760789.346301:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155840 -> 155848 00010000:00010000:0.0:1789760789.346314:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.346326:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.346335:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.346344:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.346349:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.346357:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.346376:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.346384:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1c9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79777792->79781887] (req 79777792->79781887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.346862:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9677us (9967us total) trans 4294986784 rc 0/0 00000100:00100000:0.0:1789760789.346873:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19488 00000100:00100000:0.0:1789760789.346883:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719442167233 time 719461774107 state 0x2 ref 0 00000100:00100000:0.0:1789760789.350536:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.350621:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503882752 00000100:00100000:1.0:1789760789.350631:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503882752 00000100:00100000:1.0:1789760789.350688:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719442167233 time 719461774107 state 0x0 ref 1 00000100:00100000:1.0:1789760789.350703:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719462034523 time 719461774107 state 0x1 ref 1 00000100:00100000:1.0:1789760789.350737:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719462034523 time 719475343601 state 0x0 ref 1 00000100:00100000:1.0:1789760789.350749:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ece300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503882752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.350797:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79781888, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260c8 00010000:00010000:1.0:1789760789.350806:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.350827:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.350831:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.350837:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.350850:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.357613:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986785 is committed 00002000:00010000:1.0:1789760789.357778:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79781888 -> 79785984 00002000:00010000:1.0:1789760789.357784:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155848 -> 155856 00010000:00010000:1.0:1789760789.357797:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.357810:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.357819:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.357826:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.357830:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.357839:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.357863:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.357871:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1ca3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79781888->79785983] (req 79781888->79785983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.357967:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ece300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503882752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7214us (7435us total) trans 4294986785 rc 0/0 00000100:00100000:1.0:1789760789.357977:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19489 00000100:00100000:1.0:1789760789.357987:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719462034523 time 719475343601 state 0x2 ref 0 00000100:00100000:2.0:1789760789.360805:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.360971:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503882880 00000100:00100000:0.0:1789760789.360983:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503882880 00000100:00100000:0.0:1789760789.361036:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719462034523 time 719475343601 state 0x0 ref 1 00000100:00100000:0.0:1789760789.361050:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719475604017 time 719475343601 state 0x1 ref 1 00000100:00100000:0.0:1789760789.361082:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719475604017 time 719485688156 state 0x0 ref 1 00000100:00100000:0.0:1789760789.361094:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.361213:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79785984, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260d0 00010000:00010000:0.0:1789760789.361222:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.361238:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.361305:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.361312:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.361325:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.369808:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986786 is committed 00002000:00010000:3.0:1789760789.369931:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79785984 -> 79790080 00002000:00010000:3.0:1789760789.369936:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155856 -> 155864 00010000:00010000:3.0:1789760789.369950:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.369961:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.369969:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.369977:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760789.369982:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.369991:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.370010:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.370022:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1caa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79785984->79790079] (req 79785984->79790079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.370126:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503882880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9029us (9325us total) trans 4294986786 rc 0/0 00000100:00100000:3.0:1789760789.370135:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19490 00000100:00100000:3.0:1789760789.370144:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719475604017 time 719485688156 state 0x2 ref 0 00000100:00100000:2.0:1789760789.375946:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.376081:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883008 00000100:00100000:0.0:1789760789.376090:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883008 00000100:00100000:0.0:1789760789.376144:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719475604017 time 719485688156 state 0x0 ref 1 00000100:00100000:0.0:1789760789.376159:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719485948572 time 719485688156 state 0x1 ref 1 00000100:00100000:0.0:1789760789.376191:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719485948572 time 719500797422 state 0x0 ref 1 00000100:00100000:0.0:1789760789.376203:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.376250:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79790080, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260d8 00010000:00010000:0.0:1789760789.376260:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.376276:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.376280:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.376286:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.376300:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.381012:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986787 is committed 00002000:00010000:1.0:1789760789.381125:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79790080 -> 79794176 00002000:00010000:1.0:1789760789.381131:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155864 -> 155872 00010000:00010000:1.0:1789760789.381144:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.381157:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.381166:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.381173:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.381177:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.381191:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.381211:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.381219:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f1cb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79790080->79794175] (req 79790080->79794175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.381436:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5228us (5495us total) trans 4294986787 rc 0/0 00000100:00100000:1.0:1789760789.381447:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19491 00000100:00100000:1.0:1789760789.381456:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719485948572 time 719500797422 state 0x2 ref 0 00000100:00100000:0.0:1789760789.384485:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.384567:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883136 00000100:00100000:2.0:1789760789.384576:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883136 00000100:00100000:2.0:1789760789.384640:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719485948572 time 719500797422 state 0x0 ref 1 00000100:00100000:2.0:1789760789.384654:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719501057838 time 719500797422 state 0x1 ref 1 00000100:00100000:2.0:1789760789.384683:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719501057838 time 719509289396 state 0x0 ref 1 00000100:00100000:2.0:1789760789.384694:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503883136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.384739:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79794176, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260e0 00010000:00010000:2.0:1789760789.384749:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.384765:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.384769:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.384775:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.384789:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.389694:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986788 is committed 00002000:00010000:2.0:1789760789.389819:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79794176 -> 79798272 00002000:00010000:2.0:1789760789.389824:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155872 -> 155880 00010000:00010000:2.0:1789760789.389837:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.389851:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.389860:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.389868:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.389872:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.389882:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.389901:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.389909:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1cb8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79794176->79798271] (req 79794176->79798271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.389993:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503883136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5295us (5513us total) trans 4294986788 rc 0/0 00000100:00100000:2.0:1789760789.390003:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19492 00000100:00100000:2.0:1789760789.390013:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719501057838 time 719509289396 state 0x2 ref 0 00000100:00100000:2.0:1789760789.392739:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.393090:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883264 00000100:00100000:1.0:1789760789.393101:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883264 00000100:00100000:1.0:1789760789.393167:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719501057838 time 719509289396 state 0x0 ref 1 00000100:00100000:1.0:1789760789.393181:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719509549812 time 719509289396 state 0x1 ref 1 00000100:00100000:1.0:1789760789.393248:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719509549812 time 719517853447 state 0x0 ref 1 00000100:00100000:1.0:1789760789.393259:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.393306:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79798272, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260e8 00010000:00010000:1.0:1789760789.393315:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.393331:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.393335:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.393341:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.393354:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.397537:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986789 is committed 00002000:00010000:1.0:1789760789.397646:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79798272 -> 79802368 00002000:00010000:1.0:1789760789.397652:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155880 -> 155888 00010000:00010000:1.0:1789760789.397665:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.397679:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.397687:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.397693:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.397698:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.397706:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.397730:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.397738:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79798272->79802367] (req 79798272->79802367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.397832:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4568us (5096us total) trans 4294986789 rc 0/0 00000100:00100000:1.0:1789760789.397842:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19493 00000100:00100000:1.0:1789760789.397851:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719509549812 time 719517853447 state 0x2 ref 0 00000100:00100000:1.0:1789760789.403159:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.403294:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883392 00000100:00100000:3.0:1789760789.403304:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883392 00000100:00100000:3.0:1789760789.403357:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719509549812 time 719517853447 state 0x0 ref 1 00000100:00100000:3.0:1789760789.403372:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719518113863 time 719517853447 state 0x1 ref 1 00000100:00100000:3.0:1789760789.403433:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719518113863 time 719528039233 state 0x0 ref 1 00000100:00100000:3.0:1789760789.403445:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503883392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.403488:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79802368, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260f0 00010000:00010000:3.0:1789760789.403498:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.403524:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760789.403529:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.403535:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.403553:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.407821:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986790 is committed 00002000:00010000:3.0:1789760789.408725:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79802368 -> 79806464 00002000:00010000:3.0:1789760789.408733:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155888 -> 155896 00010000:00010000:3.0:1789760789.408746:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.408759:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.408767:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.408774:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760789.408779:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.408787:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.408804:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.408811:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1cc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79802368->79806463] (req 79802368->79806463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.408912:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503883392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5464us (5758us total) trans 4294986790 rc 0/0 00000100:00100000:3.0:1789760789.408921:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19494 00000100:00100000:3.0:1789760789.408930:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719518113863 time 719528039233 state 0x2 ref 0 00000100:00100000:0.0:1789760789.411561:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.411819:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883520 00000100:00100000:2.0:1789760789.411828:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883520 00000100:00100000:2.0:1789760789.411882:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719518113863 time 719528039233 state 0x0 ref 1 00000100:00100000:2.0:1789760789.411897:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719528299649 time 719528039233 state 0x1 ref 1 00000100:00100000:2.0:1789760789.411930:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719528299649 time 719536536416 state 0x0 ref 1 00000100:00100000:2.0:1789760789.411942:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.411989:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79806464, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x260f8 00010000:00010000:2.0:1789760789.411998:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.412014:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.412018:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.412024:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.412037:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.418014:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986791 is committed 00002000:00010000:1.0:1789760789.418139:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79806464 -> 79810560 00002000:00010000:1.0:1789760789.418144:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155896 -> 155904 00010000:00010000:1.0:1789760789.418157:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.418168:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.418176:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.418183:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.418187:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.418196:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.418214:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.418221:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1ccd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79806464->79810559] (req 79806464->79810559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.418324:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7ecfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6379us (6767us total) trans 4294986791 rc 0/0 00000100:00100000:1.0:1789760789.418334:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19495 00000100:00100000:1.0:1789760789.418343:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719528299649 time 719536536416 state 0x2 ref 0 00000100:00100000:2.0:1789760789.423494:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.426344:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883648 00000100:00100000:0.0:1789760789.426362:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883648 00000100:00100000:0.0:1789760789.426446:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719528299649 time 719536536416 state 0x0 ref 1 00000100:00100000:0.0:1789760789.426463:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719536796832 time 719536536416 state 0x1 ref 1 00000100:00100000:0.0:1789760789.426496:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719536796832 time 719551103158 state 0x0 ref 1 00000100:00100000:0.0:1789760789.426520:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.426567:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79810560, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26100 00010000:00010000:0.0:1789760789.426575:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.426591:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.426606:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.426612:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.426624:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.433353:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986792 is committed 00002000:00010000:2.0:1789760789.435667:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79810560 -> 79814656 00002000:00010000:2.0:1789760789.435677:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155904 -> 155912 00010000:00010000:2.0:1789760789.435690:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435705:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435715:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435722:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.435727:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435736:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435757:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435764:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1cd4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79810560->79814655] (req 79810560->79814655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.435859:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9349us (12369us total) trans 4294986792 rc 0/0 00000100:00100000:2.0:1789760789.435869:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19496 00000100:00100000:2.0:1789760789.435880:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719536796832 time 719551103158 state 0x2 ref 0 00000100:00100000:3.0:1789760789.439944:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.441620:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883776 00000100:00100000:1.0:1789760789.441634:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883776 00000100:00100000:1.0:1789760789.441694:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719536796832 time 719551103158 state 0x0 ref 1 00000100:00100000:1.0:1789760789.441710:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719551363574 time 719551103158 state 0x1 ref 1 00000100:00100000:1.0:1789760789.441744:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719551363574 time 719566350491 state 0x0 ref 1 00000100:00100000:1.0:1789760789.441757:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503883776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.441803:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79814656, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26108 00010000:00010000:1.0:1789760789.441812:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.441828:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.441832:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.441837:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.441850:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.447883:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986793 is committed 00002000:00010000:1.0:1789760789.447992:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79814656 -> 79818752 00002000:00010000:1.0:1789760789.447997:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155912 -> 155920 00010000:00010000:1.0:1789760789.448010:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.448022:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.448031:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.448038:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.448042:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.448051:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.448074:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.448081:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1cdb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79814656->79818751] (req 79814656->79818751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.448173:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503883776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6413us (8233us total) trans 4294986793 rc 0/0 00000100:00100000:1.0:1789760789.448183:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19497 00000100:00100000:1.0:1789760789.448193:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719551363574 time 719566350491 state 0x2 ref 0 00000100:00100000:0.0:1789760789.450786:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.450931:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503883904 00000100:00100000:3.0:1789760789.450940:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503883904 00000100:00100000:3.0:1789760789.450994:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719551363574 time 719566350491 state 0x0 ref 1 00000100:00100000:3.0:1789760789.451010:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719566610907 time 719566350491 state 0x1 ref 1 00000100:00100000:3.0:1789760789.451039:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719566610907 time 719575646041 state 0x0 ref 1 00000100:00100000:3.0:1789760789.451051:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.451096:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79818752, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26110 00010000:00010000:3.0:1789760789.451105:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.451120:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760789.451124:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.451130:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.451149:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.458044:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986794 is committed 00002000:00010000:2.0:1789760789.460635:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79818752 -> 79822848 00002000:00010000:2.0:1789760789.460646:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155920 -> 155928 00010000:00010000:2.0:1789760789.460665:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.460679:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.460689:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.460697:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.460701:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.460710:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.460729:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.460737:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1ce2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79818752->79822847] (req 79818752->79822847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.460853:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503883904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9799us (10073us total) trans 4294986794 rc 0/0 00000100:00100000:2.0:1789760789.460864:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19498 00000100:00100000:2.0:1789760789.460877:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719566610907 time 719575646041 state 0x2 ref 0 00000100:00100000:2.0:1789760789.464586:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.464795:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884032 00000100:00100000:1.0:1789760789.464805:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884032 00000100:00100000:1.0:1789760789.464859:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719566610907 time 719575646041 state 0x0 ref 1 00000100:00100000:1.0:1789760789.464873:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719575906457 time 719575646041 state 0x1 ref 1 00000100:00100000:1.0:1789760789.464931:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719575906457 time 719589536598 state 0x0 ref 1 00000100:00100000:1.0:1789760789.464943:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.464996:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79822848, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26118 00010000:00010000:1.0:1789760789.465006:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.465022:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.465027:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.465034:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.465049:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.469495:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986795 is committed 00002000:00010000:1.0:1789760789.469711:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79822848 -> 79826944 00002000:00010000:1.0:1789760789.469716:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155928 -> 155936 00010000:00010000:1.0:1789760789.469730:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469742:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469751:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469758:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.469762:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469771:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469790:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469802:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1ce9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79822848->79826943] (req 79822848->79826943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.469928:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4983us (5348us total) trans 4294986795 rc 0/0 00000100:00100000:1.0:1789760789.469938:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19499 00000100:00100000:1.0:1789760789.469947:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719575906457 time 719589536598 state 0x2 ref 0 00000100:00100000:1.0:1789760789.477262:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.477334:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884160 00000100:00100000:2.0:1789760789.477346:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884160 00000100:00100000:2.0:1789760789.477483:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719575906457 time 719589536598 state 0x0 ref 1 00000100:00100000:2.0:1789760789.477498:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719589797014 time 719589536598 state 0x1 ref 1 00000100:00100000:2.0:1789760789.477572:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719589797014 time 719602178009 state 0x0 ref 1 00000100:00100000:2.0:1789760789.477585:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.477633:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79826944, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26120 00010000:00010000:2.0:1789760789.477641:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.477657:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.477661:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.477668:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.477682:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.482011:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986796 is committed 00002000:00010000:2.0:1789760789.482824:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79826944 -> 79831040 00002000:00010000:2.0:1789760789.482830:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155936 -> 155944 00010000:00010000:2.0:1789760789.482844:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.482864:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.482874:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.482881:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.482885:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.482893:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.482910:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.482916:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1cf0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79826944->79831039] (req 79826944->79831039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.483003:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5415us (5745us total) trans 4294986796 rc 0/0 00000100:00100000:2.0:1789760789.483011:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19500 00000100:00100000:2.0:1789760789.483020:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719589797014 time 719602178009 state 0x2 ref 0 00000100:00100000:0.0:1789760789.490095:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.490257:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884288 00000100:00100000:2.0:1789760789.490267:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884288 00000100:00100000:2.0:1789760789.490376:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719589797014 time 719602178009 state 0x0 ref 1 00000100:00100000:2.0:1789760789.490412:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719602438425 time 719602178009 state 0x1 ref 1 00000100:00100000:2.0:1789760789.490441:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719602438425 time 719615047825 state 0x0 ref 1 00000100:00100000:2.0:1789760789.490454:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.490501:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79831040, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26128 00010000:00010000:2.0:1789760789.490522:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.490537:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.490541:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.490547:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.490561:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.495880:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986797 is committed 00002000:00010000:2.0:1789760789.495998:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79831040 -> 79835136 00002000:00010000:2.0:1789760789.496004:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155944 -> 155952 00010000:00010000:2.0:1789760789.496016:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.496028:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.496043:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.496051:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.496056:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.496066:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.496083:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.496092:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1cf7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79831040->79835135] (req 79831040->79835135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.496224:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5767us (6133us total) trans 4294986797 rc 0/0 00000100:00100000:2.0:1789760789.496244:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19501 00000100:00100000:2.0:1789760789.496255:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719602438425 time 719615047825 state 0x2 ref 0 00000100:00100000:0.0:1789760789.501778:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.501905:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884416 00000100:00100000:3.0:1789760789.501914:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884416 00000100:00100000:3.0:1789760789.501965:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719602438425 time 719615047825 state 0x0 ref 1 00000100:00100000:3.0:1789760789.501979:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719615308241 time 719615047825 state 0x1 ref 1 00000100:00100000:3.0:1789760789.502010:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719615308241 time 719626616306 state 0x0 ref 1 00000100:00100000:3.0:1789760789.502022:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.502065:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79835136, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26130 00010000:00010000:3.0:1789760789.502075:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.502091:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760789.502095:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.502102:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.502115:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.507037:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986798 is committed 00002000:00010000:3.0:1789760789.507211:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79835136 -> 79839232 00002000:00010000:3.0:1789760789.507216:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155952 -> 155960 00010000:00010000:3.0:1789760789.507283:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.507297:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.507306:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.507313:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760789.507318:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.507327:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.507347:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.507355:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1cfe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79835136->79839231] (req 79835136->79839231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.507712:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5685us (5937us total) trans 4294986798 rc 0/0 00000100:00100000:2.0:1789760789.507735:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19502 00000100:00100000:2.0:1789760789.507746:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719615308241 time 719626616306 state 0x2 ref 0 00000100:00100000:1.0:1789760789.510928:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.511017:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884544 00000100:00100000:2.0:1789760789.511027:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884544 00000100:00100000:2.0:1789760789.511079:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719615308241 time 719626616306 state 0x0 ref 1 00000100:00100000:2.0:1789760789.511093:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719626876722 time 719626616306 state 0x1 ref 1 00000100:00100000:2.0:1789760789.511122:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719626876722 time 719635728388 state 0x0 ref 1 00000100:00100000:2.0:1789760789.511132:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc13c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.511175:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79839232, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26138 00010000:00010000:2.0:1789760789.511184:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.511199:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.511203:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.511209:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.511222:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.516701:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986799 is committed 00002000:00010000:1.0:1789760789.517137:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79839232 -> 79843328 00002000:00010000:1.0:1789760789.517144:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155960 -> 155968 00010000:00010000:1.0:1789760789.517157:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.517170:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.517179:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.517186:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.517190:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.517203:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.517221:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.517228:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1d05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79839232->79843327] (req 79839232->79843327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.517476:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc13c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6340us (6551us total) trans 4294986799 rc 0/0 00000100:00100000:1.0:1789760789.517488:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19503 00000100:00100000:1.0:1789760789.517497:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719626876722 time 719635728388 state 0x2 ref 0 00000100:00100000:0.0:1789760789.520547:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.520692:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884672 00000100:00100000:2.0:1789760789.520703:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884672 00000100:00100000:2.0:1789760789.520757:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719626876722 time 719635728388 state 0x0 ref 1 00000100:00100000:2.0:1789760789.520772:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719635988804 time 719635728388 state 0x1 ref 1 00000100:00100000:2.0:1789760789.520806:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719635988804 time 719645413197 state 0x0 ref 1 00000100:00100000:2.0:1789760789.520818:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.520865:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79843328, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26140 00010000:00010000:2.0:1789760789.520874:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.520890:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.520895:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.520901:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.520915:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.524824:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986800 is committed 00002000:00010000:2.0:1789760789.526613:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79843328 -> 79847424 00002000:00010000:2.0:1789760789.526622:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155968 -> 155976 00010000:00010000:2.0:1789760789.526636:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.526648:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.526657:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.526670:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.526675:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.526684:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.526703:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.526711:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1d0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79843328->79847423] (req 79843328->79847423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.527140:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6318us (6597us total) trans 4294986800 rc 0/0 00000100:00100000:2.0:1789760789.527152:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19504 00000100:00100000:2.0:1789760789.527163:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719635988804 time 719645413197 state 0x2 ref 0 00000100:00100000:0.0:1789760789.533693:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.536435:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884800 00000100:00100000:1.0:1789760789.536452:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884800 00000100:00100000:1.0:1789760789.536524:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719635988804 time 719645413197 state 0x0 ref 1 00000100:00100000:1.0:1789760789.536540:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719645673613 time 719645413197 state 0x1 ref 1 00000100:00100000:1.0:1789760789.536574:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719645673613 time 719661180736 state 0x0 ref 1 00000100:00100000:1.0:1789760789.536588:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.536637:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79847424, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26148 00010000:00010000:1.0:1789760789.536647:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.536664:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.536669:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.536675:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.536688:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.540721:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986801 is committed 00002000:00010000:1.0:1789760789.540836:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79847424 -> 79851520 00002000:00010000:1.0:1789760789.540841:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155976 -> 155984 00010000:00010000:1.0:1789760789.540855:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.540869:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.540878:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.540886:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.540891:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.540899:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.540923:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.540931:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79847424->79851519] (req 79847424->79851519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.541025:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503884800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4435us (7337us total) trans 4294986801 rc 0/0 00000100:00100000:1.0:1789760789.541035:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19505 00000100:00100000:1.0:1789760789.541046:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719645673613 time 719661180736 state 0x2 ref 0 00000100:00100000:1.0:1789760789.552677:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.552814:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503884928 00000100:00100000:0.0:1789760789.552824:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503884928 00000100:00100000:0.0:1789760789.552880:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719645673613 time 719661180736 state 0x0 ref 1 00000100:00100000:0.0:1789760789.552895:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719661441152 time 719661180736 state 0x1 ref 1 00000100:00100000:0.0:1789760789.552927:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719661441152 time 719677533296 state 0x0 ref 1 00000100:00100000:0.0:1789760789.552939:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.552983:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79851520, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26150 00010000:00010000:0.0:1789760789.552992:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.553014:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.553018:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.553024:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.553037:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.560702:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986802 is committed 00002000:00010000:0.0:1789760789.561342:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79851520 -> 79855616 00002000:00010000:0.0:1789760789.561348:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155984 -> 155992 00010000:00010000:0.0:1789760789.561361:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561373:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561766:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561777:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.561782:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561792:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561816:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.561825:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f1d1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79851520->79855615] (req 79851520->79855615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.561983:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503884928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9040us (9311us total) trans 4294986802 rc 0/0 00000100:00100000:0.0:1789760789.561993:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19506 00000100:00100000:0.0:1789760789.562002:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719661441152 time 719677533296 state 0x2 ref 0 00000100:00100000:0.0:1789760789.567233:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.567323:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885056 00000100:00100000:1.0:1789760789.567332:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885056 00000100:00100000:1.0:1789760789.567429:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719661441152 time 719677533296 state 0x0 ref 1 00000100:00100000:1.0:1789760789.567444:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719677793712 time 719677533296 state 0x1 ref 1 00000100:00100000:1.0:1789760789.567465:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719677793712 time 719692072553 state 0x0 ref 1 00000100:00100000:1.0:1789760789.567478:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503885056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.567541:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79855616, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26158 00010000:00010000:1.0:1789760789.567550:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.567571:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.567575:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.567581:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.567595:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.574749:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986803 is committed 00002000:00010000:1.0:1789760789.574868:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79855616 -> 79859712 00002000:00010000:1.0:1789760789.574874:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 155992 -> 156000 00010000:00010000:1.0:1789760789.574887:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.574900:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.574909:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.574916:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.574921:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.574929:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.574951:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.574958:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79855616->79859711] (req 79855616->79859711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.575043:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503885056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7563us (7816us total) trans 4294986803 rc 0/0 00000100:00100000:1.0:1789760789.575051:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19507 00000100:00100000:1.0:1789760789.575059:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719677793712 time 719692072553 state 0x2 ref 0 00000100:00100000:2.0:1789760789.577933:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.578028:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885184 00000100:00100000:0.0:1789760789.578037:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885184 00000100:00100000:0.0:1789760789.578089:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719677793712 time 719692072553 state 0x0 ref 1 00000100:00100000:0.0:1789760789.578103:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719692332969 time 719692072553 state 0x1 ref 1 00000100:00100000:1.0:1789760789.578817:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719692332969 time 719703420258 state 0x0 ref 1 00000100:00100000:1.0:1789760789.578835:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.578884:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79859712, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26160 00010000:00010000:1.0:1789760789.578893:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.578909:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.578914:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.578921:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.578935:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.584517:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986804 is committed 00002000:00010000:1.0:1789760789.585808:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79859712 -> 79863808 00002000:00010000:1.0:1789760789.585815:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156000 -> 156008 00010000:00010000:1.0:1789760789.585829:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.585842:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.585850:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.585857:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.585863:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.585872:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.585891:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.585898:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f1d28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79859712->79863807] (req 79859712->79863807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.586109:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7273us (8179us total) trans 4294986804 rc 0/0 00000100:00100000:1.0:1789760789.586120:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19508 00000100:00100000:1.0:1789760789.586129:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719692332969 time 719703420258 state 0x2 ref 0 00000100:00100000:3.0:1789760789.592771:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.593982:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885312 00000100:00100000:2.0:1789760789.593996:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885312 00000100:00100000:2.0:1789760789.594056:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719692332969 time 719703420258 state 0x0 ref 1 00000100:00100000:2.0:1789760789.594073:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719703680674 time 719703420258 state 0x1 ref 1 00000100:00100000:2.0:1789760789.594107:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719703680674 time 719718713764 state 0x0 ref 1 00000100:00100000:2.0:1789760789.594120:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503885312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.594166:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79863808, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26168 00010000:00010000:2.0:1789760789.594175:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.594191:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.594195:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.594201:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.594213:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.602329:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986805 is committed 00002000:00010000:3.0:1789760789.602644:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79863808 -> 79867904 00002000:00010000:3.0:1789760789.602650:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156008 -> 156016 00010000:00010000:3.0:1789760789.602664:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.602676:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.602685:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.602691:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760789.602695:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.602704:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.602723:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.602730:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1d2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79863808->79867903] (req 79863808->79867903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.602825:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503885312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8703us (10058us total) trans 4294986805 rc 0/0 00000100:00100000:3.0:1789760789.602835:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19509 00000100:00100000:3.0:1789760789.602846:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719703680674 time 719718713764 state 0x2 ref 0 00000100:00100000:3.0:1789760789.616715:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.616811:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885440 00000100:00100000:2.0:1789760789.616822:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885440 00000100:00100000:2.0:1789760789.616872:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719703680674 time 719718713764 state 0x0 ref 1 00000100:00100000:2.0:1789760789.616885:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719718974180 time 719718713764 state 0x1 ref 1 00000100:00100000:2.0:1789760789.616966:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719718974180 time 719741564680 state 0x0 ref 1 00000100:00100000:2.0:1789760789.616979:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.617022:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79867904, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26170 00010000:00010000:2.0:1789760789.617031:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.617047:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.617052:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.617058:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.617072:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.625688:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986806 is committed 00002000:00010000:1.0:1789760789.625888:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79867904 -> 79872000 00002000:00010000:1.0:1789760789.625893:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156016 -> 156024 00010000:00010000:1.0:1789760789.625907:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.625920:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.625928:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.625935:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.625940:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.625949:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.625968:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.625975:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1d36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79867904->79871999] (req 79867904->79871999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.626196:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9213us (9482us total) trans 4294986806 rc 0/0 00000100:00100000:1.0:1789760789.626207:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19510 00000100:00100000:1.0:1789760789.626217:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719718974180 time 719741564680 state 0x2 ref 0 00000100:00100000:2.0:1789760789.636714:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.636823:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885568 00000100:00100000:1.0:1789760789.636835:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885568 00000100:00100000:1.0:1789760789.636898:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719718974180 time 719741564680 state 0x0 ref 1 00000100:00100000:1.0:1789760789.636914:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719741825096 time 719741564680 state 0x1 ref 1 00000100:00100000:1.0:1789760789.636944:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719741825096 time 719761550862 state 0x0 ref 1 00000100:00100000:1.0:1789760789.636958:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.637010:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79872000, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26178 00010000:00010000:1.0:1789760789.637021:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.637036:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.637041:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.637048:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.637062:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.646973:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986807 is committed 00002000:00010000:1.0:1789760789.647077:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79872000 -> 79876096 00002000:00010000:1.0:1789760789.647082:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156024 -> 156032 00010000:00010000:1.0:1789760789.647095:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647107:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647115:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647122:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.647127:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647137:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647157:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647164:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79872000->79876095] (req 79872000->79876095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.647262:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10302us (10553us total) trans 4294986807 rc 0/0 00000100:00100000:1.0:1789760789.647273:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19511 00000100:00100000:1.0:1789760789.647282:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719741825096 time 719761550862 state 0x2 ref 0 00000100:00100000:1.0:1789760789.654900:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.658678:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885696 00000100:00100000:2.0:1789760789.658692:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885696 00000100:00100000:2.0:1789760789.658755:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719741825096 time 719761550862 state 0x0 ref 1 00000100:00100000:2.0:1789760789.658770:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719761811278 time 719761550862 state 0x1 ref 1 00000100:00100000:2.0:1789760789.658856:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719761811278 time 719783462382 state 0x0 ref 1 00000100:00100000:2.0:1789760789.658870:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.658918:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79876096, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26180 00010000:00010000:2.0:1789760789.658927:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.658949:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.658954:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.658960:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.658972:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.664972:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986808 is committed 00002000:00010000:1.0:1789760789.665083:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79876096 -> 79880192 00002000:00010000:1.0:1789760789.665088:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156032 -> 156040 00010000:00010000:1.0:1789760789.665104:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.665118:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.665127:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.665135:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.665140:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.665150:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.665172:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.665180:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f1d44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79876096->79880191] (req 79876096->79880191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.665741:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6866us (10843us total) trans 4294986808 rc 0/0 00000100:00100000:1.0:1789760789.665753:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19512 00000100:00100000:1.0:1789760789.665764:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719761811278 time 719783462382 state 0x2 ref 0 00000100:00100000:0.0:1789760789.674915:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.675723:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885824 00000100:00100000:1.0:1789760789.675733:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885824 00000100:00100000:1.0:1789760789.675793:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719761811278 time 719783462382 state 0x0 ref 1 00000100:00100000:1.0:1789760789.675809:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719783722798 time 719783462382 state 0x1 ref 1 00000100:00100000:1.0:1789760789.675844:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719783722798 time 719800450097 state 0x0 ref 1 00000100:00100000:1.0:1789760789.675858:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.675906:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79880192, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26188 00010000:00010000:1.0:1789760789.675915:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.675931:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.675935:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.675942:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.675955:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.681158:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986809 is committed 00002000:00010000:1.0:1789760789.682653:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79880192 -> 79884288 00002000:00010000:1.0:1789760789.682662:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156040 -> 156048 00010000:00010000:1.0:1789760789.682676:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682694:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682703:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682710:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.682715:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682725:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682744:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682752:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79880192->79884287] (req 79880192->79884287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.682861:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7000us (7949us total) trans 4294986809 rc 0/0 00000100:00100000:1.0:1789760789.682872:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19513 00000100:00100000:1.0:1789760789.682881:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719783722798 time 719800450097 state 0x2 ref 0 00000100:00100000:2.0:1789760789.690208:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.692756:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503885952 00000100:00100000:1.0:1789760789.692772:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503885952 00000100:00100000:1.0:1789760789.692834:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719783722798 time 719800450097 state 0x0 ref 1 00000100:00100000:1.0:1789760789.692850:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719800710513 time 719800450097 state 0x1 ref 1 00000100:00100000:1.0:1789760789.692888:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719800710513 time 719817494046 state 0x0 ref 1 00000100:00100000:1.0:1789760789.692901:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.692946:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79884288, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26190 00010000:00010000:1.0:1789760789.692954:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.692969:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.692974:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.692980:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.692995:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.699889:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986810 is committed 00002000:00010000:0.0:1789760789.700045:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79884288 -> 79888384 00002000:00010000:0.0:1789760789.700051:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156048 -> 156056 00010000:00010000:0.0:1789760789.700064:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.700081:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.700091:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.700098:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.700103:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.700113:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.700132:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.700205:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f1d52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79884288->79888383] (req 79884288->79888383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.700472:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503885952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7569us (10268us total) trans 4294986810 rc 0/0 00000100:00100000:0.0:1789760789.700483:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19514 00000100:00100000:0.0:1789760789.700492:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719800710513 time 719817494046 state 0x2 ref 0 00000100:00100000:2.0:1789760789.706983:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.707139:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886080 00000100:00100000:1.0:1789760789.707149:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886080 00000100:00100000:1.0:1789760789.707203:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719800710513 time 719817494046 state 0x0 ref 1 00000100:00100000:1.0:1789760789.707218:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719817754462 time 719817494046 state 0x1 ref 1 00000100:00100000:1.0:1789760789.707337:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719817754462 time 719831943167 state 0x0 ref 1 00000100:00100000:1.0:1789760789.707350:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503886080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.707413:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79888384, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x26198 00010000:00010000:1.0:1789760789.707422:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.707440:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.707444:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.707451:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.707465:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.713214:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986811 is committed 00002000:00010000:1.0:1789760789.713379:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79888384 -> 79892480 00002000:00010000:1.0:1789760789.713384:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156056 -> 156064 00010000:00010000:1.0:1789760789.713471:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.713485:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.713500:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.713518:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.713524:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.713533:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.713622:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.713631:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1d59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79888384->79892479] (req 79888384->79892479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.714612:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503886080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7255us (7630us total) trans 4294986811 rc 0/0 00000100:00100000:1.0:1789760789.714626:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19515 00000100:00100000:1.0:1789760789.714637:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719817754462 time 719831943167 state 0x2 ref 0 00000100:00100000:0.0:1789760789.720933:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.721016:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886208 00000100:00100000:2.0:1789760789.721026:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886208 00000100:00100000:2.0:1789760789.721080:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719817754462 time 719831943167 state 0x0 ref 1 00000100:00100000:2.0:1789760789.721096:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719832203583 time 719831943167 state 0x1 ref 1 00000100:00100000:2.0:1789760789.721159:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719832203583 time 719845764568 state 0x0 ref 1 00000100:00100000:2.0:1789760789.721171:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503886208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.721215:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79892480, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x261a0 00010000:00010000:2.0:1789760789.721224:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.721257:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.721262:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.721268:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.721282:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.728706:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986812 is committed 00002000:00010000:2.0:1789760789.728856:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79892480 -> 79896576 00002000:00010000:2.0:1789760789.728861:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156064 -> 156072 00010000:00010000:2.0:1789760789.728874:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.728887:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.728896:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.728903:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.728908:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.728917:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.728941:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.728949:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1d60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79892480->79896575] (req 79892480->79896575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.729080:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503886208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7906us (8153us total) trans 4294986812 rc 0/0 00000100:00100000:2.0:1789760789.729090:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19516 00000100:00100000:2.0:1789760789.729100:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719832203583 time 719845764568 state 0x2 ref 0 00000100:00100000:2.0:1789760789.734099:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.734186:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886336 00000100:00100000:3.0:1789760789.734196:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886336 00000100:00100000:3.0:1789760789.734248:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719832203583 time 719845764568 state 0x0 ref 1 00000100:00100000:3.0:1789760789.734263:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719846024984 time 719845764568 state 0x1 ref 1 00000100:00100000:3.0:1789760789.734295:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719846024984 time 719858901718 state 0x0 ref 1 00000100:00100000:3.0:1789760789.734308:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503886336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.734352:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79896576, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x261a8 00010000:00010000:3.0:1789760789.734361:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.734377:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760789.734381:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.734423:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.734438:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.740209:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986813 is committed 00002000:00010000:2.0:1789760789.740908:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79896576 -> 79900672 00002000:00010000:2.0:1789760789.740914:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156072 -> 156080 00010000:00010000:2.0:1789760789.740928:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.740941:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.740950:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.740958:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.740963:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.740972:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.740992:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.741000:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1d67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79896576->79900671] (req 79896576->79900671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.741665:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503886336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7075us (7292us total) trans 4294986813 rc 0/0 00000100:00100000:2.0:1789760789.741678:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19517 00000100:00100000:2.0:1789760789.741688:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719846024984 time 719858901718 state 0x2 ref 0 00000100:00100000:1.0:1789760789.745557:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.745773:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886464 00000100:00100000:0.0:1789760789.745784:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886464 00000100:00100000:0.0:1789760789.745837:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719846024984 time 719858901718 state 0x0 ref 1 00000100:00100000:0.0:1789760789.745852:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719859162134 time 719858901718 state 0x1 ref 1 00000100:00100000:0.0:1789760789.745884:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719859162134 time 719870490938 state 0x0 ref 1 00000100:00100000:0.0:1789760789.745897:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503886464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.745940:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79900672, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x261b0 00010000:00010000:0.0:1789760789.745949:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.745964:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760789.745968:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.745975:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.745989:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.750869:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986814 is committed 00002000:00010000:0.0:1789760789.750978:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79900672 -> 79904768 00002000:00010000:0.0:1789760789.750983:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156080 -> 156088 00010000:00010000:0.0:1789760789.750996:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.751015:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.751023:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.751030:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.751035:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.751044:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.751064:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.751073:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1d6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79900672->79904767] (req 79900672->79904767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.751160:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503886464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5262us (5608us total) trans 4294986814 rc 0/0 00000100:00100000:0.0:1789760789.751170:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19518 00000100:00100000:0.0:1789760789.751180:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719859162134 time 719870490938 state 0x2 ref 0 00000100:00100000:0.0:1789760789.758079:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.758176:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886592 00000100:00100000:1.0:1789760789.758186:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886592 00000100:00100000:1.0:1789760789.758242:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719859162134 time 719870490938 state 0x0 ref 1 00000100:00100000:1.0:1789760789.758260:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719870751354 time 719870490938 state 0x1 ref 1 00000100:00100000:1.0:1789760789.758286:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719870751354 time 719882893034 state 0x0 ref 1 00000100:00100000:1.0:1789760789.758297:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503886592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.758339:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79904768, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x261b8 00010000:00010000:1.0:1789760789.758348:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.758364:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.758369:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.758374:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.758418:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.764909:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986815 is committed 00002000:00010000:1.0:1789760789.765025:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79904768 -> 79908864 00002000:00010000:1.0:1789760789.765030:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156088 -> 156096 00010000:00010000:1.0:1789760789.765042:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.765055:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.765063:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.765071:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.765076:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.765085:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.765104:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.765111:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79904768->79908863] (req 79904768->79908863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.765206:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503886592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6905us (7150us total) trans 4294986815 rc 0/0 00000100:00100000:1.0:1789760789.765216:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19519 00000100:00100000:1.0:1789760789.765225:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719870751354 time 719882893034 state 0x2 ref 0 00000100:00100000:2.0:1789760789.773504:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.773614:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886720 00000100:00100000:1.0:1789760789.773623:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886720 00000100:00100000:1.0:1789760789.773980:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719870751354 time 719882893034 state 0x0 ref 1 00000100:00100000:1.0:1789760789.773995:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719883153450 time 719882893034 state 0x1 ref 1 00000100:00100000:1.0:1789760789.774026:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719883153450 time 719898632449 state 0x0 ref 1 00000100:00100000:1.0:1789760789.774044:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503886720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.774091:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79908864, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x261c0 00010000:00010000:1.0:1789760789.774100:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.774116:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.774120:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.774125:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.774139:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.778090:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986816 is committed 00002000:00010000:1.0:1789760789.778187:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79908864 -> 79912960 00002000:00010000:1.0:1789760789.778193:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156096 -> 156104 00010000:00010000:1.0:1789760789.778205:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.778217:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.778236:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.778245:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.778251:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.778259:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.778279:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.778287:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79908864->79912959] (req 79908864->79912959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.778376:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503886720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4330us (4878us total) trans 4294986816 rc 0/0 00000100:00100000:1.0:1789760789.778386:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19520 00000100:00100000:1.0:1789760789.778436:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719883153450 time 719898632449 state 0x2 ref 0 00000100:00100000:2.0:1789760789.783689:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.784408:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886848 00000100:00100000:1.0:1789760789.784419:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886848 00000100:00100000:1.0:1789760789.784475:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719883153450 time 719898632449 state 0x0 ref 1 00000100:00100000:1.0:1789760789.784490:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719898892865 time 719898632449 state 0x1 ref 1 00000100:00100000:1.0:1789760789.784530:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719898892865 time 719909136250 state 0x0 ref 1 00000100:00100000:1.0:1789760789.784543:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503886848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.784596:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79912960, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x261c8 00010000:00010000:1.0:1789760789.784606:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.784621:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.784625:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.784632:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.784656:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.789379:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986817 is committed 00002000:00010000:1.0:1789760789.789523:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79912960 -> 79917056 00002000:00010000:1.0:1789760789.789528:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:1.0:1789760789.789530:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00002000:00010000:1.0:1789760789.789533:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156104 -> 156112 00010000:00010000:1.0:1789760789.789553:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.789567:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.789576:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.789583:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.789587:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.789601:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.789621:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.789629:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1d83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79912960->79917055] (req 79912960->79917055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.789719:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503886848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5173us (6039us total) trans 4294986817 rc 0/0 00000100:00100000:1.0:1789760789.789730:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19521 00000100:00100000:1.0:1789760789.789739:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719898892865 time 719909136250 state 0x2 ref 0 00000100:00100000:0.0:1789760789.793049:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.793125:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503886976 00000100:00100000:2.0:1789760789.793135:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503886976 00000100:00100000:2.0:1789760789.793192:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719898892865 time 719909136250 state 0x0 ref 1 00000100:00100000:2.0:1789760789.793209:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719909396666 time 719909136250 state 0x1 ref 1 00000100:00100000:2.0:1789760789.794728:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719909396666 time 719919325914 state 0x0 ref 1 00000100:00100000:2.0:1789760789.794756:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503886976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.794812:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79917056, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x261d0 00010000:00010000:2.0:1789760789.794821:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.794837:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.794841:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.794847:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.794860:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.810368:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986818 is committed 00002000:00010000:2.0:1789760789.810499:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79917056 -> 79921152 00002000:00010000:2.0:1789760789.810504:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156112 -> 156120 00010000:00010000:2.0:1789760789.810526:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.810537:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.810553:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.810560:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.810565:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.810574:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.810591:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.810606:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79917056->79921151] (req 79917056->79921151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.810689:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503886976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15937us (17645us total) trans 4294986818 rc 0/0 00000100:00100000:2.0:1789760789.810699:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19522 00000100:00100000:2.0:1789760789.810708:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719909396666 time 719919325914 state 0x2 ref 0 00000100:00100000:2.0:1789760789.816631:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.816756:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503887104 00000100:00100000:1.0:1789760789.816767:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503887104 00000100:00100000:1.0:1789760789.816822:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719909396666 time 719919325914 state 0x0 ref 1 00000100:00100000:1.0:1789760789.816836:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719919586330 time 719919325914 state 0x1 ref 1 00000100:00100000:1.0:1789760789.816869:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719919586330 time 719941476216 state 0x0 ref 1 00000100:00100000:1.0:1789760789.816882:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.816931:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79921152, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x261d8 00010000:00010000:1.0:1789760789.816940:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.816955:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760789.816961:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.816967:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.816981:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.827042:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986819 is committed 00002000:00010000:1.0:1789760789.827164:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79921152 -> 79925248 00002000:00010000:1.0:1789760789.827170:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156120 -> 156128 00010000:00010000:1.0:1789760789.827184:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.827197:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.827208:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.827215:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.827220:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.827229:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.827273:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.827281:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1d91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79921152->79925247] (req 79921152->79925247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.827376:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10490us (10748us total) trans 4294986819 rc 0/0 00000100:00100000:1.0:1789760789.827386:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19523 00000100:00100000:1.0:1789760789.827490:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719919586330 time 719941476216 state 0x2 ref 0 00000100:00100000:1.0:1789760789.833416:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.833488:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503887232 00000100:00100000:2.0:1789760789.833499:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503887232 00000100:00100000:2.0:1789760789.833570:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719919586330 time 719941476216 state 0x0 ref 1 00000100:00100000:2.0:1789760789.833585:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719941736632 time 719941476216 state 0x1 ref 1 00000100:00100000:2.0:1789760789.833742:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719941736632 time 719958347199 state 0x0 ref 1 00000100:00100000:2.0:1789760789.833755:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.833797:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79925248, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x261e0 00010000:00010000:2.0:1789760789.833806:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.833822:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.833826:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.833832:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.833846:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.840101:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986820 is committed 00002000:00010000:2.0:1789760789.840223:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79925248 -> 79929344 00002000:00010000:2.0:1789760789.840236:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156128 -> 156136 00010000:00010000:2.0:1789760789.840249:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.840261:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.840270:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.840278:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.840282:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.840299:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.840318:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.840325:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79925248->79929343] (req 79925248->79929343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.840447:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6690us (7057us total) trans 4294986820 rc 0/0 00000100:00100000:2.0:1789760789.840458:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19524 00000100:00100000:2.0:1789760789.840468:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719941736632 time 719958347199 state 0x2 ref 0 00000100:00100000:0.0:1789760789.851049:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.851464:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503887360 00000100:00100000:2.0:1789760789.851474:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503887360 00000100:00100000:2.0:1789760789.851539:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719941736632 time 719958347199 state 0x0 ref 1 00000100:00100000:2.0:1789760789.851563:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719958607615 time 719958347199 state 0x1 ref 1 00000100:00100000:2.0:1789760789.851592:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719958607615 time 719976199079 state 0x0 ref 1 00000100:00100000:2.0:1789760789.851605:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.851648:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79929344, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x261e8 00010000:00010000:2.0:1789760789.851656:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.851672:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.851677:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.851683:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.851698:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.857496:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986821 is committed 00002000:00010000:2.0:1789760789.857820:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79929344 -> 79933440 00002000:00010000:2.0:1789760789.857825:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156136 -> 156144 00010000:00010000:2.0:1789760789.857838:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.857851:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.857861:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.857869:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.857874:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.857883:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.857915:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.857924:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1d9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79929344->79933439] (req 79929344->79933439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.858055:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6446us (7009us total) trans 4294986821 rc 0/0 00000100:00100000:2.0:1789760789.858066:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19525 00000100:00100000:2.0:1789760789.858075:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719958607615 time 719976199079 state 0x2 ref 0 00000100:00100000:0.0:1789760789.865017:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.865166:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503887488 00000100:00100000:2.0:1789760789.865176:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503887488 00000100:00100000:2.0:1789760789.865230:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719958607615 time 719976199079 state 0x0 ref 1 00000100:00100000:2.0:1789760789.865256:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719976459495 time 719976199079 state 0x1 ref 1 00000100:00100000:2.0:1789760789.865286:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719976459495 time 719989893405 state 0x0 ref 1 00000100:00100000:2.0:1789760789.865298:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.865344:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79933440, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x261f0 00010000:00010000:2.0:1789760789.865354:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.865369:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760789.865385:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.865424:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.865439:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.873452:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986822 is committed 00002000:00010000:2.0:1789760789.873730:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79933440 -> 79937536 00002000:00010000:2.0:1789760789.873736:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156144 -> 156152 00010000:00010000:2.0:1789760789.873751:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.873766:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.873776:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.873784:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.873789:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.873798:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.873816:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.873829:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1da6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79933440->79937535] (req 79933440->79937535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.873929:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8628us (8917us total) trans 4294986822 rc 0/0 00000100:00100000:2.0:1789760789.873939:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19526 00000100:00100000:2.0:1789760789.873950:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719976459495 time 719989893405 state 0x2 ref 0 00000100:00100000:1.0:1789760789.883685:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.889180:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503887616 00000100:00100000:2.0:1789760789.889195:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503887616 00000100:00100000:2.0:1789760789.889272:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719976459495 time 719989893405 state 0x0 ref 1 00000100:00100000:2.0:1789760789.889288:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719990153821 time 719989893405 state 0x1 ref 1 00000100:00100000:2.0:1789760789.889322:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719990153821 time 720013928637 state 0x0 ref 1 00000100:00100000:2.0:1789760789.889336:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.889404:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79937536, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x261f8 00010000:00010000:2.0:1789760789.889413:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.889431:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760789.889435:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.889440:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.889461:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.901492:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986823 is committed 00002000:00010000:2.0:1789760789.901766:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79937536 -> 79941632 00002000:00010000:2.0:1789760789.901772:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156152 -> 156160 00010000:00010000:2.0:1789760789.901785:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.901798:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.901807:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.901815:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.901819:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.901827:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.901845:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.901854:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1dad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79937536->79941631] (req 79937536->79941631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.901998:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12660us (18319us total) trans 4294986823 rc 0/0 00000100:00100000:2.0:1789760789.902009:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19527 00000100:00100000:2.0:1789760789.902018:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719990153821 time 720013928637 state 0x2 ref 0 00000100:00100000:0.0:1789760789.914603:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.914701:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503887744 00000100:00100000:1.0:1789760789.914711:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503887744 00000100:00100000:1.0:1789760789.914771:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 719990153821 time 720013928637 state 0x0 ref 1 00000100:00100000:1.0:1789760789.914790:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720014189053 time 720013928637 state 0x1 ref 1 00000100:00100000:1.0:1789760789.914822:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720014189053 time 720039430011 state 0x0 ref 1 00000100:00100000:1.0:1789760789.914833:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.914885:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79941632, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26200 00010000:00010000:1.0:1789760789.914893:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.914909:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760789.914914:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.914921:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.914933:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.921014:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986824 is committed 00002000:00010000:1.0:1789760789.921130:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79941632 -> 79945728 00002000:00010000:1.0:1789760789.921136:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156160 -> 156168 00010000:00010000:1.0:1789760789.921149:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.921161:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.921170:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.921178:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760789.921182:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.921191:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.921215:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.921224:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1db4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79941632->79945727] (req 79941632->79945727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.921357:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6520us (6757us total) trans 4294986824 rc 0/0 00000100:00100000:1.0:1789760789.921368:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19528 00000100:00100000:1.0:1789760789.921378:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720014189053 time 720039430011 state 0x2 ref 0 00000100:00100000:0.0:1789760789.925768:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.926214:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503887872 00000100:00100000:3.0:1789760789.926223:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503887872 00000100:00100000:3.0:1789760789.926290:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720014189053 time 720039430011 state 0x0 ref 1 00000100:00100000:3.0:1789760789.926305:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720039690427 time 720039430011 state 0x1 ref 1 00000100:00100000:3.0:1789760789.926333:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720039690427 time 720050940371 state 0x0 ref 1 00000100:00100000:3.0:1789760789.926345:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.926519:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79945728, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26208 00010000:00010000:3.0:1789760789.926529:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.926545:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760789.926643:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.926651:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.926667:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.932613:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986825 is committed 00002000:00010000:2.0:1789760789.932730:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79945728 -> 79949824 00002000:00010000:2.0:1789760789.932736:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156168 -> 156176 00010000:00010000:2.0:1789760789.932748:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.932761:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.932770:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.932777:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.932782:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.932792:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.932812:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.932824:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1dbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79945728->79949823] (req 79945728->79949823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.932986:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503887872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6637us (7220us total) trans 4294986825 rc 0/0 00000100:00100000:2.0:1789760789.932998:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19529 00000100:00100000:2.0:1789760789.933007:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720039690427 time 720050940371 state 0x2 ref 0 00000100:00100000:2.0:1789760789.944859:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760789.945037:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888000 00000100:00100000:0.0:1789760789.945046:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888000 00000100:00100000:0.0:1789760789.945173:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720039690427 time 720050940371 state 0x0 ref 1 00000100:00100000:0.0:1789760789.945187:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720051200787 time 720050940371 state 0x1 ref 1 00000100:00100000:0.0:1789760789.945260:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720051200787 time 720069866628 state 0x0 ref 1 00000100:00100000:0.0:1789760789.945281:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760789.945327:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79949824, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26210 00010000:00010000:0.0:1789760789.945336:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.945351:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760789.945355:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.945363:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.945377:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.951157:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986826 is committed 00002000:00010000:0.0:1789760789.951601:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79949824 -> 79953920 00002000:00010000:0.0:1789760789.951606:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156176 -> 156184 00010000:00010000:0.0:1789760789.951621:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.951634:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.951643:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.951649:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760789.951653:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.951662:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.951681:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.951695:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1dc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79949824->79953919] (req 79949824->79953919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.951793:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6517us (6942us total) trans 4294986826 rc 0/0 00000100:00100000:0.0:1789760789.951803:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19530 00000100:00100000:0.0:1789760789.951814:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720051200787 time 720069866628 state 0x2 ref 0 00000100:00100000:0.0:1789760789.964300:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760789.964480:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888128 00000100:00100000:2.0:1789760789.964489:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888128 00000100:00100000:2.0:1789760789.964561:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720051200787 time 720069866628 state 0x0 ref 1 00000100:00100000:2.0:1789760789.964575:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720070127044 time 720069866628 state 0x1 ref 1 00000100:00100000:2.0:1789760789.964608:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720070127044 time 720089214425 state 0x0 ref 1 00000100:00100000:2.0:1789760789.964619:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503888128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760789.964664:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79953920, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26218 00010000:00010000:2.0:1789760789.964673:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.964688:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760789.964693:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.964699:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.964712:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.972772:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986827 is committed 00002000:00010000:2.0:1789760789.972880:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79953920 -> 79958016 00002000:00010000:2.0:1789760789.972885:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156184 -> 156192 00010000:00010000:2.0:1789760789.972899:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.972912:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.972921:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.972928:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.972938:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.972947:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.972967:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.972978:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1dc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79953920->79958015] (req 79953920->79958015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.973079:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503888128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8455us (8783us total) trans 4294986827 rc 0/0 00000100:00100000:2.0:1789760789.973089:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19531 00000100:00100000:2.0:1789760789.973098:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720070127044 time 720089214425 state 0x2 ref 0 00000100:00100000:3.0:1789760789.975467:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760789.975533:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888256 00000100:00100000:1.0:1789760789.975543:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888256 00000100:00100000:1.0:1789760789.975594:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720070127044 time 720089214425 state 0x0 ref 1 00000100:00100000:1.0:1789760789.975608:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720089474841 time 720089214425 state 0x1 ref 1 00000100:00100000:1.0:1789760789.975685:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720089474841 time 720100290974 state 0x0 ref 1 00000100:00100000:1.0:1789760789.975697:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760789.975741:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79958016, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26220 00010000:00010000:1.0:1789760789.975749:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.975765:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760789.975771:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.975777:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.975792:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.985705:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986828 is committed 00002000:00010000:2.0:1789760789.993663:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79958016 -> 79962112 00002000:00010000:2.0:1789760789.993673:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156192 -> 156200 00010000:00010000:2.0:1789760789.993688:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.993702:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.993711:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.993719:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760789.993725:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.993735:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.993761:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.993769:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1dd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79958016->79962111] (req 79958016->79962111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.993874:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18174us (18410us total) trans 4294986828 rc 0/0 00000100:00100000:2.0:1789760789.993884:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19532 00000100:00100000:2.0:1789760789.993895:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720089474841 time 720100290974 state 0x2 ref 0 00000100:00100000:1.0:1789760789.998520:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760789.998627:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888384 00000100:00100000:3.0:1789760789.998637:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888384 00000100:00100000:3.0:1789760789.998689:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720089474841 time 720100290974 state 0x0 ref 1 00000100:00100000:3.0:1789760789.998704:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720100551390 time 720100290974 state 0x1 ref 1 00000100:00100000:3.0:1789760789.998734:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720100551390 time 720123340200 state 0x0 ref 1 00000100:00100000:3.0:1789760789.998746:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503888384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760789.998795:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79962112, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26228 00010000:00010000:3.0:1789760789.998805:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.998821:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760789.998826:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.998832:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.998845:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.004834:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986829 is committed 00002000:00010000:3.0:1789760790.004960:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79962112 -> 79966208 00002000:00010000:3.0:1789760790.004966:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156200 -> 156208 00010000:00010000:3.0:1789760790.004979:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.004992:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.005001:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.005008:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.005013:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.005021:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.005040:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.005048:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79962112->79966207] (req 79962112->79966207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.005612:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503888384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6861us (7104us total) trans 4294986829 rc 0/0 00000100:00100000:3.0:1789760790.005624:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19533 00000100:00100000:3.0:1789760790.005633:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720100551390 time 720123340200 state 0x2 ref 0 00000100:00100000:3.0:1789760790.011552:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760790.011759:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888512 00000100:00100000:0.0:1789760790.011769:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888512 00000100:00100000:0.0:1789760790.011838:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720100551390 time 720123340200 state 0x0 ref 1 00000100:00100000:0.0:1789760790.011853:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720123600616 time 720123340200 state 0x1 ref 1 00000100:00100000:0.0:1789760790.011884:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720123600616 time 720136490878 state 0x0 ref 1 00000100:00100000:0.0:1789760790.011897:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.011941:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79966208, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26230 00010000:00010000:0.0:1789760790.011951:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.011974:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760790.011979:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.011985:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.011998:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760790.012009:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760790.018656:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986830 is committed 00002000:00010000:3.0:1789760790.021457:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79966208 -> 79970304 00002000:00010000:3.0:1789760790.021467:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156208 -> 156216 00010000:00010000:3.0:1789760790.021482:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.021495:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.021512:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.021521:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.021526:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.021535:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.021554:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.021562:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f1dde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79966208->79970303] (req 79966208->79970303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.021701:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9801us (10152us total) trans 4294986830 rc 0/0 00000100:00100000:3.0:1789760790.021712:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19534 00000100:00100000:3.0:1789760790.021722:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720123600616 time 720136490878 state 0x2 ref 0 00000100:00100000:0.0:1789760790.029500:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.029700:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888640 00000100:00100000:2.0:1789760790.029710:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888640 00000100:00100000:2.0:1789760790.029764:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720123600616 time 720136490878 state 0x0 ref 1 00000100:00100000:2.0:1789760790.029778:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720136751294 time 720136490878 state 0x1 ref 1 00000100:00100000:2.0:1789760790.029861:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720136751294 time 720154466306 state 0x0 ref 1 00000100:00100000:2.0:1789760790.029875:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503888640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.029923:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79970304, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26238 00010000:00010000:2.0:1789760790.029932:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.029953:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.029957:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.029963:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.029976:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.034526:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760790.035364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986831 is committed 00002000:00010000:1.0:1789760790.036176:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79970304 -> 79974400 00002000:00010000:1.0:1789760790.036183:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156216 -> 156224 00010000:00010000:1.0:1789760790.036197:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.036209:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.036217:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.036224:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.036228:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.036249:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.036269:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.036287:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f1de5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79970304->79974399] (req 79970304->79974399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.036379:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503888640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6501us (6884us total) trans 4294986831 rc 0/0 00000100:00100000:1.0:1789760790.036432:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19535 00000100:00100000:1.0:1789760790.036443:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720136751294 time 720154466306 state 0x2 ref 0 00000100:00100000:1.0:1789760790.038519:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.038605:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888768 00000100:00100000:2.0:1789760790.038614:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888768 00000100:00100000:2.0:1789760790.038666:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720136751294 time 720154466306 state 0x0 ref 1 00000100:00100000:2.0:1789760790.038682:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720154726722 time 720154466306 state 0x1 ref 1 00000100:00100000:2.0:1789760790.038750:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720154726722 time 720163356547 state 0x0 ref 1 00000100:00100000:2.0:1789760790.038763:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.038813:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79974400, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26240 00010000:00010000:2.0:1789760790.038823:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.038838:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.038842:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.038849:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.038867:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.046042:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986832 is committed 00002000:00010000:1.0:1789760790.046253:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79974400 -> 79978496 00002000:00010000:1.0:1789760790.046259:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156224 -> 156232 00010000:00010000:1.0:1789760790.046272:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.046285:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.046296:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.046303:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.046309:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.046318:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.046337:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.046346:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f1dec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79974400->79978495] (req 79974400->79978495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.046515:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7742us (7999us total) trans 4294986832 rc 0/0 00000100:00100000:1.0:1789760790.046525:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19536 00000100:00100000:1.0:1789760790.046535:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720154726722 time 720163356547 state 0x2 ref 0 00000100:00100000:3.0:1789760790.050805:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.051750:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503888896 00000100:00100000:2.0:1789760790.051764:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503888896 00000100:00100000:2.0:1789760790.051878:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720154726722 time 720163356547 state 0x0 ref 1 00000100:00100000:2.0:1789760790.051894:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720163616963 time 720163356547 state 0x1 ref 1 00000100:00100000:2.0:1789760790.051966:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720163616963 time 720176572363 state 0x0 ref 1 00000100:00100000:2.0:1789760790.051979:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.052029:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79978496, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26248 00010000:00010000:2.0:1789760790.052038:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.052055:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.052060:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.052066:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.052079:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.058515:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986833 is committed 00002000:00010000:2.0:1789760790.058962:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79978496 -> 79982592 00002000:00010000:2.0:1789760790.058968:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156232 -> 156240 00010000:00010000:2.0:1789760790.058983:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.058997:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.059006:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.059014:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.059019:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.059029:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.059051:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.059060:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1df3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79978496->79982591] (req 79978496->79982591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.059152:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503888896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7171us (8352us total) trans 4294986833 rc 0/0 00000100:00100000:2.0:1789760790.059164:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19537 00000100:00100000:2.0:1789760790.059173:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720163616963 time 720176572363 state 0x2 ref 0 00000100:00100000:0.0:1789760790.062481:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.064231:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889024 00000100:00100000:3.0:1789760790.064245:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889024 00000100:00100000:3.0:1789760790.064305:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720163616963 time 720176572363 state 0x0 ref 1 00000100:00100000:3.0:1789760790.064321:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720176832779 time 720176572363 state 0x1 ref 1 00000100:00100000:3.0:1789760790.064354:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720176832779 time 720188960308 state 0x0 ref 1 00000100:00100000:3.0:1789760790.064367:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.064426:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79982592, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26250 00010000:00010000:3.0:1789760790.064436:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064452:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.064457:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.064473:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064488:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.070636:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986834 is committed 00002000:00010000:3.0:1789760790.070748:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79982592 -> 79986688 00002000:00010000:3.0:1789760790.070752:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156240 -> 156248 00010000:00010000:3.0:1789760790.070766:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.070778:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.070788:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.070803:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.070809:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.070819:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.070840:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.070847:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1dfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79982592->79986687] (req 79982592->79986687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.070980:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6610us (8502us total) trans 4294986834 rc 0/0 00000100:00100000:3.0:1789760790.070991:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19538 00000100:00100000:3.0:1789760790.071000:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720176832779 time 720188960308 state 0x2 ref 0 00000100:00100000:1.0:1789760790.077484:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.080064:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889152 00000100:00100000:2.0:1789760790.080078:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889152 00000100:00100000:2.0:1789760790.080137:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720176832779 time 720188960308 state 0x0 ref 1 00000100:00100000:2.0:1789760790.080151:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720189220724 time 720188960308 state 0x1 ref 1 00000100:00100000:2.0:1789760790.080184:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720189220724 time 720204790457 state 0x0 ref 1 00000100:00100000:2.0:1789760790.080197:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.080246:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79986688, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26258 00010000:00010000:2.0:1789760790.080255:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.080270:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.080275:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.080280:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.080294:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.084784:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986835 is committed 00002000:00010000:1.0:1789760790.085001:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79986688 -> 79990784 00002000:00010000:1.0:1789760790.085007:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156248 -> 156256 00010000:00010000:1.0:1789760790.085019:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.085030:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.085039:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.085047:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.085052:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.085061:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.085080:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.085088:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f1e01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79986688->79990783] (req 79986688->79990783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.085210:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5011us (7730us total) trans 4294986835 rc 0/0 00000100:00100000:1.0:1789760790.085221:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19539 00000100:00100000:1.0:1789760790.085285:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720189220724 time 720204790457 state 0x2 ref 0 00000100:00100000:0.0:1789760790.093437:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.093590:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889280 00000100:00100000:1.0:1789760790.093610:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889280 00000100:00100000:1.0:1789760790.093665:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720189220724 time 720204790457 state 0x0 ref 1 00000100:00100000:1.0:1789760790.093680:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720205050873 time 720204790457 state 0x1 ref 1 00000100:00100000:1.0:1789760790.093710:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720205050873 time 720218317021 state 0x0 ref 1 00000100:00100000:1.0:1789760790.093723:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.093774:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79990784, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26260 00010000:00010000:1.0:1789760790.093783:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.093917:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.093922:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.093928:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.093943:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.098708:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986836 is committed 00002000:00010000:1.0:1789760790.108025:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79990784 -> 79994880 00002000:00010000:1.0:1789760790.108037:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156256 -> 156264 00010000:00010000:1.0:1789760790.108053:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.108066:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.108076:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.108084:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.108089:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.108097:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.108118:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.108125:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79990784->79994879] (req 79990784->79994879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.108229:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14503us (14836us total) trans 4294986836 rc 0/0 00000100:00100000:1.0:1789760790.108239:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19540 00000100:00100000:1.0:1789760790.108248:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720205050873 time 720218317021 state 0x2 ref 0 00000100:00100000:0.0:1789760790.112877:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.113645:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889408 00000100:00100000:3.0:1789760790.113656:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889408 00000100:00100000:3.0:1789760790.113714:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720205050873 time 720218317021 state 0x0 ref 1 00000100:00100000:3.0:1789760790.113730:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720218577437 time 720218317021 state 0x1 ref 1 00000100:00100000:3.0:1789760790.113762:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720218577437 time 720238368434 state 0x0 ref 1 00000100:00100000:3.0:1789760790.113775:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.113823:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79994880, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26268 00010000:00010000:3.0:1789760790.113833:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.113849:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.113853:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.113860:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.113874:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.122441:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986837 is committed 00002000:00010000:2.0:1789760790.124739:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79994880 -> 79998976 00002000:00010000:2.0:1789760790.124748:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156264 -> 156272 00010000:00010000:2.0:1789760790.124761:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.124773:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.124781:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.124789:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.124793:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.124803:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.124821:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.124829:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79994880->79998975] (req 79994880->79998975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.125051:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11273us (12179us total) trans 4294986837 rc 0/0 00000100:00100000:2.0:1789760790.125061:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19541 00000100:00100000:2.0:1789760790.125071:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720218577437 time 720238368434 state 0x2 ref 0 00000100:00100000:2.0:1789760790.132560:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.134447:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889536 00000100:00100000:3.0:1789760790.134461:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889536 00000100:00100000:2.0:1789760790.134701:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720218577437 time 720238368434 state 0x0 ref 1 00000100:00100000:2.0:1789760790.134717:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720238628850 time 720238368434 state 0x1 ref 1 00000100:00100000:2.0:1789760790.134805:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720238628850 time 720259409768 state 0x0 ref 1 00000100:00100000:2.0:1789760790.134818:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.134866:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 79998976, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26270 00010000:00010000:2.0:1789760790.134875:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.134891:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.134895:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.134901:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.134914:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.147453:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986838 is committed 00002000:00010000:2.0:1789760790.149960:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 79998976 -> 80003072 00002000:00010000:2.0:1789760790.149969:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156272 -> 156280 00010000:00010000:2.0:1789760790.149983:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.149995:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.150010:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.150023:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.150032:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.150044:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.150079:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.150092:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1e16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [79998976->80003071] (req 79998976->80003071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.150426:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15567us (17833us total) trans 4294986838 rc 0/0 00000100:00100000:2.0:1789760790.150437:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19542 00000100:00100000:2.0:1789760790.150745:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720238628850 time 720259409768 state 0x2 ref 0 00000100:00100000:0.0:1789760790.154445:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.154580:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889664 00000100:00100000:1.0:1789760790.154591:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889664 00000100:00100000:1.0:1789760790.154647:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720238628850 time 720259409768 state 0x0 ref 1 00000100:00100000:1.0:1789760790.154662:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720259670184 time 720259409768 state 0x1 ref 1 00000100:00100000:1.0:1789760790.154696:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720259670184 time 720279302338 state 0x0 ref 1 00000100:00100000:1.0:1789760790.154709:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503889664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.154763:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80003072, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26278 00010000:00010000:1.0:1789760790.154772:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.154787:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.154793:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.154799:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.154814:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.166317:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986839 is committed 00002000:00010000:1.0:1789760790.166614:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80003072 -> 80007168 00002000:00010000:1.0:1789760790.166619:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156280 -> 156288 00010000:00010000:1.0:1789760790.166632:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.166644:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.166653:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.166659:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.166663:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.166672:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.166689:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.166701:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80003072->80007167] (req 80003072->80007167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.166797:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503889664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12085us (12356us total) trans 4294986839 rc 0/0 00000100:00100000:1.0:1789760790.166807:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19543 00000100:00100000:1.0:1789760790.166817:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720259670184 time 720279302338 state 0x2 ref 0 00000100:00100000:0.0:1789760790.170821:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.170897:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889792 00000100:00100000:1.0:1789760790.170907:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889792 00000100:00100000:1.0:1789760790.170974:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720259670184 time 720279302338 state 0x0 ref 1 00000100:00100000:1.0:1789760790.170989:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720279562754 time 720279302338 state 0x1 ref 1 00000100:00100000:1.0:1789760790.171011:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720279562754 time 720295618458 state 0x0 ref 1 00000100:00100000:1.0:1789760790.171024:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.171072:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80007168, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26280 00010000:00010000:1.0:1789760790.171081:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.171097:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.171102:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.171109:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.171128:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.193460:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986840 is committed 00002000:00010000:1.0:1789760790.193604:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80007168 -> 80011264 00002000:00010000:1.0:1789760790.193609:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156288 -> 156296 00010000:00010000:1.0:1789760790.193623:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.193634:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.193643:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.193650:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.193654:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.193663:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.193682:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.193689:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80007168->80011263] (req 80007168->80011263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.193791:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503889792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 22764us (22975us total) trans 4294986840 rc 0/0 00000100:00100000:1.0:1789760790.193803:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19544 00000100:00100000:1.0:1789760790.193812:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720279562754 time 720295618458 state 0x2 ref 0 00000100:00100000:2.0:1789760790.201727:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.201979:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503889920 00000100:00100000:3.0:1789760790.201990:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503889920 00000100:00100000:3.0:1789760790.202043:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720279562754 time 720295618458 state 0x0 ref 1 00000100:00100000:3.0:1789760790.202059:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720295878874 time 720295618458 state 0x1 ref 1 00000100:00100000:3.0:1789760790.202094:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720295878874 time 720326700313 state 0x0 ref 1 00000100:00100000:3.0:1789760790.202107:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503889920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.202152:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80011264, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26288 00010000:00010000:3.0:1789760790.202162:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.202190:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.202195:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.202202:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.202216:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.211562:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986841 is committed 00002000:00010000:3.0:1789760790.212104:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80011264 -> 80015360 00002000:00010000:3.0:1789760790.212110:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156296 -> 156304 00010000:00010000:3.0:1789760790.212123:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212135:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212143:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212150:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.212155:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212164:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212475:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212489:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80011264->80015359] (req 80011264->80015359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.212973:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503889920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10863us (11251us total) trans 4294986841 rc 0/0 00000100:00100000:3.0:1789760790.212985:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19545 00000100:00100000:3.0:1789760790.212995:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720295878874 time 720326700313 state 0x2 ref 0 00000100:00100000:0.0:1789760790.217757:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.217899:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890048 00000100:00100000:2.0:1789760790.217909:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890048 00000100:00100000:2.0:1789760790.217963:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720295878874 time 720326700313 state 0x0 ref 1 00000100:00100000:2.0:1789760790.217978:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720326960729 time 720326700313 state 0x1 ref 1 00000100:00100000:2.0:1789760790.218013:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720326960729 time 720342619713 state 0x0 ref 1 00000100:00100000:2.0:1789760790.218025:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503890048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.218071:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80015360, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26290 00010000:00010000:2.0:1789760790.218080:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.218096:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.218102:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.218108:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.218122:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.231114:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986842 is committed 00002000:00010000:2.0:1789760790.231255:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80015360 -> 80019456 00002000:00010000:2.0:1789760790.231261:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156304 -> 156312 00010000:00010000:2.0:1789760790.231281:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.231294:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.231309:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.231316:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.231322:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.231332:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.231351:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.231357:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80015360->80019455] (req 80015360->80019455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.231516:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503890048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13477us (13753us total) trans 4294986842 rc 0/0 00000100:00100000:2.0:1789760790.231526:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19546 00000100:00100000:2.0:1789760790.231535:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720326960729 time 720342619713 state 0x2 ref 0 00000100:00100000:0.0:1789760790.239006:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.239103:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890176 00000100:00100000:1.0:1789760790.239114:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890176 00000100:00100000:1.0:1789760790.239168:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720326960729 time 720342619713 state 0x0 ref 1 00000100:00100000:1.0:1789760790.239196:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720342880129 time 720342619713 state 0x1 ref 1 00000100:00100000:1.0:1789760790.239225:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720342880129 time 720363831498 state 0x0 ref 1 00000100:00100000:1.0:1789760790.239471:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503890176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.239529:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80019456, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26298 00010000:00010000:1.0:1789760790.239538:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.239563:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.239568:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.239574:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.239587:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.250206:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986843 is committed 00002000:00010000:3.0:1789760790.254179:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80019456 -> 80023552 00002000:00010000:3.0:1789760790.254188:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156312 -> 156320 00010000:00010000:3.0:1789760790.254202:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.254215:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.254224:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.254243:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.254248:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.254258:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.254612:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.254621:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f1e39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80019456->80023551] (req 80019456->80023551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.254755:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503890176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15284us (15754us total) trans 4294986843 rc 0/0 00000100:00100000:3.0:1789760790.254766:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19547 00000100:00100000:3.0:1789760790.254776:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720342880129 time 720363831498 state 0x2 ref 0 00000100:00100000:3.0:1789760790.259672:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.259774:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890304 00000100:00100000:1.0:1789760790.259784:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890304 00000100:00100000:1.0:1789760790.259840:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720342880129 time 720363831498 state 0x0 ref 1 00000100:00100000:1.0:1789760790.259857:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720364091914 time 720363831498 state 0x1 ref 1 00000100:00100000:1.0:1789760790.259893:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720364091914 time 720384499587 state 0x0 ref 1 00000100:00100000:1.0:1789760790.259905:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503890304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.259952:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80023552, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262a0 00010000:00010000:1.0:1789760790.259961:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.259977:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.259981:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.259987:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.260000:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.271435:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986844 is committed 00002000:00010000:1.0:1789760790.271627:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80023552 -> 80027648 00002000:00010000:1.0:1789760790.271632:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156320 -> 156328 00010000:00010000:1.0:1789760790.271645:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.271663:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.271672:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.271679:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.271683:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.271692:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.271711:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.271719:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80023552->80027647] (req 80023552->80027647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.271812:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503890304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11904us (12144us total) trans 4294986844 rc 0/0 00000100:00100000:1.0:1789760790.271823:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19548 00000100:00100000:1.0:1789760790.271833:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720364091914 time 720384499587 state 0x2 ref 0 00000100:00100000:0.0:1789760790.276350:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.276873:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890432 00000100:00100000:3.0:1789760790.276883:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890432 00000100:00100000:3.0:1789760790.276935:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720364091914 time 720384499587 state 0x0 ref 1 00000100:00100000:3.0:1789760790.276949:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720384760003 time 720384499587 state 0x1 ref 1 00000100:00100000:3.0:1789760790.276986:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720384760003 time 720401592284 state 0x0 ref 1 00000100:00100000:3.0:1789760790.276999:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503890432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.277046:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80027648, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262a8 00010000:00010000:3.0:1789760790.277056:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.277077:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.277082:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.277089:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.277103:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.288306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986845 is committed 00002000:00010000:2.0:1789760790.288498:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80027648 -> 80031744 00002000:00010000:2.0:1789760790.288513:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156328 -> 156336 00010000:00010000:2.0:1789760790.288527:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.288539:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.288609:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.288616:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.288620:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.288629:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.288649:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.288657:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f1e47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80027648->80031743] (req 80027648->80031743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.288757:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503890432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11755us (12412us total) trans 4294986845 rc 0/0 00000100:00100000:2.0:1789760790.288766:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19549 00000100:00100000:2.0:1789760790.288776:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720384760003 time 720401592284 state 0x2 ref 0 00000100:00100000:0.0:1789760790.294081:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.294164:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890560 00000100:00100000:2.0:1789760790.294183:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890560 00000100:00100000:2.0:1789760790.294238:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720384760003 time 720401592284 state 0x0 ref 1 00000100:00100000:2.0:1789760790.294264:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720401852700 time 720401592284 state 0x1 ref 1 00000100:00100000:2.0:1789760790.294636:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720401852700 time 720419240126 state 0x0 ref 1 00000100:00100000:2.0:1789760790.294651:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503890560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.294701:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80031744, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262b0 00010000:00010000:2.0:1789760790.294711:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.294727:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.294732:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.294739:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.294753:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.305083:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986846 is committed 00002000:00010000:2.0:1789760790.305363:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80031744 -> 80035840 00002000:00010000:2.0:1789760790.305369:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156336 -> 156344 00010000:00010000:2.0:1789760790.305382:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.305487:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.305512:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.305521:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.305526:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.305537:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.305649:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.305658:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f1e4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80031744->80035839] (req 80031744->80035839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.305940:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503890560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11287us (11872us total) trans 4294986846 rc 0/0 00000100:00100000:2.0:1789760790.305950:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19550 00000100:00100000:2.0:1789760790.305960:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720401852700 time 720419240126 state 0x2 ref 0 00000100:00100000:2.0:1789760790.309860:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.309941:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890688 00000100:00100000:3.0:1789760790.309950:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890688 00000100:00100000:3.0:1789760790.310002:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720401852700 time 720419240126 state 0x0 ref 1 00000100:00100000:3.0:1789760790.310017:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720419500542 time 720419240126 state 0x1 ref 1 00000100:00100000:3.0:1789760790.310072:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720419500542 time 720434678348 state 0x0 ref 1 00000100:00100000:3.0:1789760790.310086:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503890688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.310131:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80035840, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262b8 00010000:00010000:3.0:1789760790.310139:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.310155:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.310159:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.310165:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.310188:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.321386:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986847 is committed 00002000:00010000:1.0:1789760790.323157:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80035840 -> 80039936 00002000:00010000:1.0:1789760790.323336:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156344 -> 156352 00010000:00010000:1.0:1789760790.323351:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.323364:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.323372:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.323379:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.323383:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.323469:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.323492:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.323499:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f1e55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80035840->80039935] (req 80035840->80039935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.324162:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503890688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14071us (14303us total) trans 4294986847 rc 0/0 00000100:00100000:1.0:1789760790.324184:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19551 00000100:00100000:1.0:1789760790.324193:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720419500542 time 720434678348 state 0x2 ref 0 00000100:00100000:0.0:1789760790.332120:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.336307:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890816 00000100:00100000:1.0:1789760790.336324:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890816 00000100:00100000:1.0:1789760790.336405:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720419500542 time 720434678348 state 0x0 ref 1 00000100:00100000:1.0:1789760790.336430:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720434938764 time 720434678348 state 0x1 ref 1 00000100:00100000:1.0:1789760790.336463:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720434938764 time 720461069514 state 0x0 ref 1 00000100:00100000:1.0:1789760790.336477:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503890816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.336535:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80039936, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262c0 00010000:00010000:1.0:1789760790.336555:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.336570:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.336574:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.336581:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.336594:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.343448:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986848 is committed 00002000:00010000:1.0:1789760790.343645:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80039936 -> 80044032 00002000:00010000:1.0:1789760790.343652:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156352 -> 156360 00010000:00010000:1.0:1789760790.343664:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.343681:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.343690:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.343697:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.343702:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.343711:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.343731:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.343739:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f1e5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80039936->80044031] (req 80039936->80044031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.343839:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503890816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7360us (11724us total) trans 4294986848 rc 0/0 00000100:00100000:1.0:1789760790.343850:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19552 00000100:00100000:1.0:1789760790.343861:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720434938764 time 720461069514 state 0x2 ref 0 00000100:00100000:0.0:1789760790.348833:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.351317:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503890944 00000100:00100000:2.0:1789760790.351332:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503890944 00000100:00100000:2.0:1789760790.351808:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720434938764 time 720461069514 state 0x0 ref 1 00000100:00100000:2.0:1789760790.351830:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720461329930 time 720461069514 state 0x1 ref 1 00000100:00100000:2.0:1789760790.351868:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720461329930 time 720476474695 state 0x0 ref 1 00000100:00100000:2.0:1789760790.351882:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503890944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.351927:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80044032, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262c8 00010000:00010000:2.0:1789760790.351936:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.351951:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.351957:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.351963:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.351976:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.357940:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986849 is committed 00002000:00010000:2.0:1789760790.358105:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80044032 -> 80048128 00002000:00010000:2.0:1789760790.358111:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156360 -> 156368 00010000:00010000:2.0:1789760790.358127:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358139:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358148:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358157:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.358162:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358235:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358258:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358267:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80044032->80048127] (req 80044032->80048127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.358367:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503890944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6483us (9537us total) trans 4294986849 rc 0/0 00000100:00100000:2.0:1789760790.358377:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19553 00000100:00100000:2.0:1789760790.358386:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720461329930 time 720476474695 state 0x2 ref 0 00000100:00100000:2.0:1789760790.362381:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760790.362729:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891072 00000100:00100000:0.0:1789760790.362739:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891072 00000100:00100000:0.0:1789760790.362792:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720461329930 time 720476474695 state 0x0 ref 1 00000100:00100000:0.0:1789760790.362806:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720476735111 time 720476474695 state 0x1 ref 1 00000100:00100000:0.0:1789760790.363035:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720476735111 time 720487639958 state 0x0 ref 1 00000100:00100000:0.0:1789760790.363048:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503891072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.363332:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80048128, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262d0 00010000:00010000:0.0:1789760790.363343:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.363361:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760790.363365:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.363373:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.364470:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.371598:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986850 is committed 00002000:00010000:0.0:1789760790.371810:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80048128 -> 80052224 00002000:00010000:0.0:1789760790.371815:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156368 -> 156376 00010000:00010000:0.0:1789760790.371829:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371842:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371852:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371860:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.371865:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371875:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371895:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.371903:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1e6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80048128->80052223] (req 80048128->80052223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.373412:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503891072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10335us (11010us total) trans 4294986850 rc 0/0 00000100:00100000:0.0:1789760790.373427:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19554 00000100:00100000:0.0:1789760790.373437:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720476735111 time 720487639958 state 0x2 ref 0 00000100:00100000:0.0:1789760790.380349:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.380691:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891200 00000100:00100000:1.0:1789760790.380701:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891200 00000100:00100000:1.0:1789760790.380759:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720476735111 time 720487639958 state 0x0 ref 1 00000100:00100000:1.0:1789760790.380774:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720487900374 time 720487639958 state 0x1 ref 1 00000100:00100000:1.0:1789760790.380806:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720487900374 time 720505412240 state 0x0 ref 1 00000100:00100000:1.0:1789760790.380818:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503891200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.380864:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80052224, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262d8 00010000:00010000:1.0:1789760790.380872:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.380888:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.380892:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.380899:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.380913:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.390837:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986851 is committed 00002000:00010000:2.0:1789760790.391027:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80052224 -> 80056320 00002000:00010000:2.0:1789760790.391033:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156376 -> 156384 00010000:00010000:2.0:1789760790.391046:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.391058:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.391079:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.391086:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.391091:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.391099:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.391118:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.391125:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f1e71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80052224->80056319] (req 80052224->80056319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.391221:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503891200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10400us (10877us total) trans 4294986851 rc 0/0 00000100:00100000:2.0:1789760790.391232:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19555 00000100:00100000:2.0:1789760790.391242:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720487900374 time 720505412240 state 0x2 ref 0 00000100:00100000:1.0:1789760790.401495:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.401596:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891328 00000100:00100000:3.0:1789760790.401606:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891328 00000100:00100000:3.0:1789760790.401660:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720487900374 time 720505412240 state 0x0 ref 1 00000100:00100000:3.0:1789760790.401674:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720505672656 time 720505412240 state 0x1 ref 1 00000100:00100000:3.0:1789760790.401707:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720505672656 time 720526314248 state 0x0 ref 1 00000100:00100000:3.0:1789760790.401720:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.401768:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80056320, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262e0 00010000:00010000:3.0:1789760790.401779:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.401795:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.401800:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.401806:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.401821:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.413036:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986852 is committed 00002000:00010000:3.0:1789760790.419455:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80056320 -> 80060416 00002000:00010000:3.0:1789760790.419467:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156384 -> 156392 00010000:00010000:3.0:1789760790.419482:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.419497:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.419519:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.419527:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.419534:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.419543:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.419562:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.419570:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80056320->80060415] (req 80056320->80060415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.419671:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 17948us (18181us total) trans 4294986852 rc 0/0 00000100:00100000:3.0:1789760790.419681:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19556 00000100:00100000:3.0:1789760790.419690:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720505672656 time 720526314248 state 0x2 ref 0 00000100:00100000:3.0:1789760790.436419:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.436521:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891456 00000100:00100000:2.0:1789760790.436532:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891456 00000100:00100000:2.0:1789760790.436589:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720505672656 time 720526314248 state 0x0 ref 1 00000100:00100000:2.0:1789760790.436603:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720526574664 time 720526314248 state 0x1 ref 1 00000100:00100000:2.0:1789760790.436635:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720526574664 time 720561241734 state 0x0 ref 1 00000100:00100000:2.0:1789760790.436647:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.436687:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80060416, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262e8 00010000:00010000:2.0:1789760790.436697:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.436713:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.436717:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.436724:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.436738:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.446649:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986853 is committed 00002000:00010000:2.0:1789760790.446820:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80060416 -> 80064512 00002000:00010000:2.0:1789760790.446826:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156392 -> 156400 00010000:00010000:2.0:1789760790.446840:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.446852:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.446860:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.446873:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.446879:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.446887:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.446909:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.446917:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80060416->80064511] (req 80060416->80064511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.447515:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10854us (11121us total) trans 4294986853 rc 0/0 00000100:00100000:2.0:1789760790.447526:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19557 00000100:00100000:2.0:1789760790.447599:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720526574664 time 720561241734 state 0x2 ref 0 00000100:00100000:0.0:1789760790.458488:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.458589:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891584 00000100:00100000:2.0:1789760790.458599:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891584 00000100:00100000:2.0:1789760790.458655:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720526574664 time 720561241734 state 0x0 ref 1 00000100:00100000:2.0:1789760790.458670:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720561502150 time 720561241734 state 0x1 ref 1 00000100:00100000:2.0:1789760790.458697:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720561502150 time 720583304725 state 0x0 ref 1 00000100:00100000:2.0:1789760790.458711:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.458760:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80064512, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262f0 00010000:00010000:2.0:1789760790.458769:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.458789:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.458793:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.458800:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.458813:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.465265:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986854 is committed 00002000:00010000:2.0:1789760790.465484:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80064512 -> 80068608 00002000:00010000:2.0:1789760790.465489:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156400 -> 156408 00010000:00010000:2.0:1789760790.465515:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465527:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465536:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465543:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.465549:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465757:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465778:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.465787:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80064512->80068607] (req 80064512->80068607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.465896:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7180us (7411us total) trans 4294986854 rc 0/0 00000100:00100000:2.0:1789760790.465909:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19558 00000100:00100000:2.0:1789760790.465919:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720561502150 time 720583304725 state 0x2 ref 0 00000100:00100000:1.0:1789760790.473620:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.473795:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891712 00000100:00100000:2.0:1789760790.473805:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891712 00000100:00100000:2.0:1789760790.473858:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720561502150 time 720583304725 state 0x0 ref 1 00000100:00100000:2.0:1789760790.473884:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720583565141 time 720583304725 state 0x1 ref 1 00000100:00100000:2.0:1789760790.473923:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720583565141 time 720598529331 state 0x0 ref 1 00000100:00100000:2.0:1789760790.473935:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.473986:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80068608, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x262f8 00010000:00010000:2.0:1789760790.473995:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.474010:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.474015:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.474022:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.474034:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.480477:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986855 is committed 00002000:00010000:2.0:1789760790.480991:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80068608 -> 80072704 00002000:00010000:2.0:1789760790.480996:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156408 -> 156416 00010000:00010000:2.0:1789760790.481010:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.481025:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.481033:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.481040:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.481044:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.481053:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.481072:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.481079:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f1e8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80068608->80072703] (req 80068608->80072703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.481240:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7292us (7615us total) trans 4294986855 rc 0/0 00000100:00100000:2.0:1789760790.481251:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19559 00000100:00100000:2.0:1789760790.481263:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720583565141 time 720598529331 state 0x2 ref 0 00000100:00100000:3.0:1789760790.489857:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.490117:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891840 00000100:00100000:1.0:1789760790.490127:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891840 00000100:00100000:1.0:1789760790.490180:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720583565141 time 720598529331 state 0x0 ref 1 00000100:00100000:1.0:1789760790.490195:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720598789747 time 720598529331 state 0x1 ref 1 00000100:00100000:1.0:1789760790.490238:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720598789747 time 720614844696 state 0x0 ref 1 00000100:00100000:1.0:1789760790.490252:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc10050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503891840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.490307:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80072704, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26300 00010000:00010000:1.0:1789760790.490317:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.490335:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.490339:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.490345:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.490358:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.502830:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986856 is committed 00002000:00010000:3.0:1789760790.502992:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80072704 -> 80076800 00002000:00010000:3.0:1789760790.502998:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156416 -> 156424 00010000:00010000:3.0:1789760790.503012:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.503024:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.503033:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.503040:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.503045:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.503054:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.503073:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.503082:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f1e94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80072704->80076799] (req 80072704->80076799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.503183:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc10050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503891840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12928us (13330us total) trans 4294986856 rc 0/0 00000100:00100000:3.0:1789760790.503193:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19560 00000100:00100000:3.0:1789760790.503202:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720598789747 time 720614844696 state 0x2 ref 0 00000100:00100000:0.0:1789760790.508565:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.508788:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503891968 00000100:00100000:3.0:1789760790.508796:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503891968 00000100:00100000:3.0:1789760790.508858:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720598789747 time 720614844696 state 0x0 ref 1 00000100:00100000:3.0:1789760790.508873:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720615105112 time 720614844696 state 0x1 ref 1 00000100:00100000:3.0:1789760790.508903:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720615105112 time 720633510050 state 0x0 ref 1 00000100:00100000:3.0:1789760790.508916:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.508960:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80076800, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26308 00010000:00010000:3.0:1789760790.508970:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.508984:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.508988:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.508994:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.509008:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.518337:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986857 is committed 00002000:00010000:3.0:1789760790.518573:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80076800 -> 80080896 00002000:00010000:3.0:1789760790.518579:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156424 -> 156432 00010000:00010000:3.0:1789760790.518593:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.518607:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.518617:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.518625:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.518630:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.518640:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.518661:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.518669:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1e9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80076800->80080895] (req 80076800->80080895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.518822:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503891968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9903us (10262us total) trans 4294986857 rc 0/0 00000100:00100000:3.0:1789760790.518832:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19561 00000100:00100000:3.0:1789760790.518841:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720615105112 time 720633510050 state 0x2 ref 0 00000100:00100000:0.0:1789760790.528989:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.529178:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892096 00000100:00100000:2.0:1789760790.529189:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892096 00000100:00100000:2.0:1789760790.529309:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720615105112 time 720633510050 state 0x0 ref 1 00000100:00100000:2.0:1789760790.529326:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720633770466 time 720633510050 state 0x1 ref 1 00000100:00100000:2.0:1789760790.529414:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720633770466 time 720654020300 state 0x0 ref 1 00000100:00100000:2.0:1789760790.529427:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.529475:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80080896, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26310 00010000:00010000:2.0:1789760790.529484:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.529499:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.529514:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.529520:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.529534:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.542852:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986858 is committed 00002000:00010000:2.0:1789760790.545986:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80080896 -> 80084992 00002000:00010000:2.0:1789760790.545996:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156432 -> 156440 00010000:00010000:2.0:1789760790.546010:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546151:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546161:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546169:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.546175:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546185:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546208:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546221:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ea2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80080896->80084991] (req 80080896->80084991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.546805:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 17374us (17820us total) trans 4294986858 rc 0/0 00000100:00100000:2.0:1789760790.546817:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19562 00000100:00100000:2.0:1789760790.546827:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720633770466 time 720654020300 state 0x2 ref 0 00000100:00100000:0.0:1789760790.551900:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.551992:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892224 00000100:00100000:3.0:1789760790.552001:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892224 00000100:00100000:3.0:1789760790.552056:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720633770466 time 720654020300 state 0x0 ref 1 00000100:00100000:3.0:1789760790.552082:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720654280716 time 720654020300 state 0x1 ref 1 00000100:00100000:3.0:1789760790.552111:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720654280716 time 720676717688 state 0x0 ref 1 00000100:00100000:3.0:1789760790.552124:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503892224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.552169:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80084992, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26318 00010000:00010000:3.0:1789760790.552178:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.552195:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.552199:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.552205:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.552219:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.562702:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986859 is committed 00002000:00010000:3.0:1789760790.563371:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80084992 -> 80089088 00002000:00010000:3.0:1789760790.563376:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156440 -> 156448 00010000:00010000:3.0:1789760790.563426:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.563446:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.563455:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.563462:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.563467:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.563475:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.563494:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.563512:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1ea9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80084992->80089087] (req 80084992->80089087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.563772:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503892224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11645us (11875us total) trans 4294986859 rc 0/0 00000100:00100000:3.0:1789760790.563782:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19563 00000100:00100000:3.0:1789760790.563793:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720654280716 time 720676717688 state 0x2 ref 0 00000100:00100000:0.0:1789760790.580673:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.580760:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892352 00000100:00100000:3.0:1789760790.580782:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892352 00000100:00100000:3.0:1789760790.580839:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720654280716 time 720676717688 state 0x0 ref 1 00000100:00100000:3.0:1789760790.580854:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720676978104 time 720676717688 state 0x1 ref 1 00000100:00100000:3.0:1789760790.580887:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720676978104 time 720705494180 state 0x0 ref 1 00000100:00100000:3.0:1789760790.580900:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.580952:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80089088, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26320 00010000:00010000:3.0:1789760790.580962:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.580980:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.580984:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.580991:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.581006:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.589512:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986860 is committed 00002000:00010000:3.0:1789760790.589634:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80089088 -> 80093184 00002000:00010000:3.0:1789760790.589639:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156448 -> 156456 00010000:00010000:3.0:1789760790.589653:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.589666:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.589681:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.589688:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.589693:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.589702:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.589721:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.589729:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80089088->80093183] (req 80089088->80093183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.589841:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8939us (9180us total) trans 4294986860 rc 0/0 00000100:00100000:3.0:1789760790.589851:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19564 00000100:00100000:3.0:1789760790.589860:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720676978104 time 720705494180 state 0x2 ref 0 00000100:00100000:0.0:1789760790.597272:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.597429:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892480 00000100:00100000:3.0:1789760790.597439:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892480 00000100:00100000:3.0:1789760790.597493:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720676978104 time 720705494180 state 0x0 ref 1 00000100:00100000:3.0:1789760790.597518:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720705754596 time 720705494180 state 0x1 ref 1 00000100:00100000:3.0:1789760790.597547:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720705754596 time 720722154217 state 0x0 ref 1 00000100:00100000:3.0:1789760790.597560:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503892480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.597607:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80093184, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26328 00010000:00010000:3.0:1789760790.597617:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.597632:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.597636:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.597642:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.597655:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.604941:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986861 is committed 00002000:00010000:2.0:1789760790.607357:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80093184 -> 80097280 00002000:00010000:2.0:1789760790.607366:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156456 -> 156464 00010000:00010000:2.0:1789760790.607379:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.607477:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.607485:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.607492:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.607497:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.607517:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.607542:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.607551:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f1eb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80093184->80097279] (req 80093184->80097279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.607723:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503892480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10160us (10455us total) trans 4294986861 rc 0/0 00000100:00100000:2.0:1789760790.607733:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19565 00000100:00100000:2.0:1789760790.607741:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720705754596 time 720722154217 state 0x2 ref 0 00000100:00100000:0.0:1789760790.616992:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.617120:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892608 00000100:00100000:2.0:1789760790.617130:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892608 00000100:00100000:2.0:1789760790.617186:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720705754596 time 720722154217 state 0x0 ref 1 00000100:00100000:2.0:1789760790.617200:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720722414633 time 720722154217 state 0x1 ref 1 00000100:00100000:2.0:1789760790.617241:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720722414633 time 720741846908 state 0x0 ref 1 00000100:00100000:2.0:1789760790.617254:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.617459:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80097280, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26330 00010000:00010000:2.0:1789760790.617469:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.617485:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.617490:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.617497:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.617524:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.629973:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986862 is committed 00002000:00010000:2.0:1789760790.630068:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80097280 -> 80101376 00002000:00010000:2.0:1789760790.630073:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156464 -> 156472 00010000:00010000:2.0:1789760790.630086:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.630099:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.630107:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.630114:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.630120:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.630128:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.630147:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.630160:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ebe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80097280->80101375] (req 80097280->80101375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.630256:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12998us (13268us total) trans 4294986862 rc 0/0 00000100:00100000:2.0:1789760790.630266:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19566 00000100:00100000:2.0:1789760790.630276:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720722414633 time 720741846908 state 0x2 ref 0 00000100:00100000:0.0:1789760790.637251:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.637519:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892736 00000100:00100000:2.0:1789760790.637528:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892736 00000100:00100000:2.0:1789760790.637592:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720722414633 time 720741846908 state 0x0 ref 1 00000100:00100000:2.0:1789760790.637607:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720742107324 time 720741846908 state 0x1 ref 1 00000100:00100000:2.0:1789760790.637637:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720742107324 time 720762243910 state 0x0 ref 1 00000100:00100000:2.0:1789760790.637651:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.637697:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80101376, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26338 00010000:00010000:2.0:1789760790.637706:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.637722:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.637726:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.637733:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.637751:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.643662:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986863 is committed 00002000:00010000:2.0:1789760790.643820:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80101376 -> 80105472 00002000:00010000:2.0:1789760790.643825:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156472 -> 156480 00010000:00010000:2.0:1789760790.643845:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.643858:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.643867:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.643875:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.643879:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.643888:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.643907:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.643915:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ec5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80101376->80105471] (req 80101376->80105471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.644011:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6358us (6764us total) trans 4294986863 rc 0/0 00000100:00100000:2.0:1789760790.644021:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19567 00000100:00100000:2.0:1789760790.644030:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720742107324 time 720762243910 state 0x2 ref 0 00000100:00100000:0.0:1789760790.647620:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.647740:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892864 00000100:00100000:3.0:1789760790.647750:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892864 00000100:00100000:3.0:1789760790.647800:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720742107324 time 720762243910 state 0x0 ref 1 00000100:00100000:3.0:1789760790.647815:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720762504326 time 720762243910 state 0x1 ref 1 00000100:00100000:3.0:1789760790.647840:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720762504326 time 720772447078 state 0x0 ref 1 00000100:00100000:3.0:1789760790.647852:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.647897:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80105472, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26340 00010000:00010000:3.0:1789760790.647907:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.647923:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.647927:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.647934:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.647948:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.653735:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986864 is committed 00002000:00010000:3.0:1789760790.653916:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80105472 -> 80109568 00002000:00010000:3.0:1789760790.653921:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156480 -> 156488 00010000:00010000:3.0:1789760790.653933:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.653945:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.653953:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.653960:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.653965:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.653978:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.653998:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.654005:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ecc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80105472->80109567] (req 80105472->80109567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.654094:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503892864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6239us (6478us total) trans 4294986864 rc 0/0 00000100:00100000:3.0:1789760790.654104:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19568 00000100:00100000:3.0:1789760790.654114:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720762504326 time 720772447078 state 0x2 ref 0 00000100:00100000:0.0:1789760790.659151:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.659457:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503892992 00000100:00100000:2.0:1789760790.659467:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503892992 00000100:00100000:2.0:1789760790.659530:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720762504326 time 720772447078 state 0x0 ref 1 00000100:00100000:2.0:1789760790.659550:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720772707494 time 720772447078 state 0x1 ref 1 00000100:00100000:2.0:1789760790.659582:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720772707494 time 720784188703 state 0x0 ref 1 00000100:00100000:2.0:1789760790.659594:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503892992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.659641:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80109568, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26348 00010000:00010000:2.0:1789760790.659650:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.659665:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.659669:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.659676:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.659689:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.665218:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986865 is committed 00002000:00010000:2.0:1789760790.665370:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80109568 -> 80113664 00002000:00010000:2.0:1789760790.665375:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156488 -> 156496 00010000:00010000:2.0:1789760790.665723:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665737:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665747:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665754:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.665758:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665768:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665786:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665794:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ed3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80109568->80113663] (req 80109568->80113663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.665921:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503892992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6324us (6773us total) trans 4294986865 rc 0/0 00000100:00100000:2.0:1789760790.665932:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19569 00000100:00100000:2.0:1789760790.665942:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720772707494 time 720784188703 state 0x2 ref 0 00000100:00100000:0.0:1789760790.671519:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.671788:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503893120 00000100:00100000:2.0:1789760790.671798:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503893120 00000100:00100000:2.0:1789760790.671852:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720772707494 time 720784188703 state 0x0 ref 1 00000100:00100000:2.0:1789760790.671867:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720784449119 time 720784188703 state 0x1 ref 1 00000100:00100000:2.0:1789760790.671897:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720784449119 time 720796503954 state 0x0 ref 1 00000100:00100000:2.0:1789760790.671909:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503893120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.671954:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80113664, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26350 00010000:00010000:2.0:1789760790.671962:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.671978:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.671982:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.671989:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.672002:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.676319:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986866 is committed 00002000:00010000:2.0:1789760790.680867:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80113664 -> 80117760 00002000:00010000:2.0:1789760790.680877:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156496 -> 156504 00010000:00010000:2.0:1789760790.680893:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.680905:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.680914:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.680921:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.680926:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.680935:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.680954:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.680962:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80113664->80117759] (req 80113664->80117759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.681106:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503893120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9193us (9600us total) trans 4294986866 rc 0/0 00000100:00100000:2.0:1789760790.681117:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19570 00000100:00100000:2.0:1789760790.681127:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720784449119 time 720796503954 state 0x2 ref 0 00000100:00100000:0.0:1789760790.685154:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.685676:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503893248 00000100:00100000:2.0:1789760790.685688:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503893248 00000100:00100000:2.0:1789760790.685755:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720784449119 time 720796503954 state 0x0 ref 1 00000100:00100000:2.0:1789760790.685770:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720796764370 time 720796503954 state 0x1 ref 1 00000100:00100000:2.0:1789760790.685800:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720796764370 time 720810406797 state 0x0 ref 1 00000100:00100000:2.0:1789760790.685811:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503893248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.685858:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80117760, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26358 00010000:00010000:2.0:1789760790.685867:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685883:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.685887:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.685894:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685907:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.692072:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986867 is committed 00002000:00010000:2.0:1789760790.692188:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80117760 -> 80121856 00002000:00010000:2.0:1789760790.692194:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156504 -> 156512 00010000:00010000:2.0:1789760790.692207:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692236:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692245:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692252:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.692257:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692266:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692286:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.692295:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1ee1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80117760->80121855] (req 80117760->80121855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.693176:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503893248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6574us (7249us total) trans 4294986867 rc 0/0 00000100:00100000:2.0:1789760790.693191:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19571 00000100:00100000:2.0:1789760790.693201:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720796764370 time 720810406797 state 0x2 ref 0 00000100:00100000:0.0:1789760790.696640:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.704554:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503893376 00000100:00100000:3.0:1789760790.704573:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503893376 00000100:00100000:3.0:1789760790.704656:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720796764370 time 720810406797 state 0x0 ref 1 00000100:00100000:3.0:1789760790.704672:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720810667213 time 720810406797 state 0x1 ref 1 00000100:00100000:3.0:1789760790.704822:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720810667213 time 720829427467 state 0x0 ref 1 00000100:00100000:3.0:1789760790.704836:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503893376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.704882:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80121856, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26360 00010000:00010000:3.0:1789760790.704891:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.704908:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760790.704912:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.704918:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.704932:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.715707:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986868 is committed 00002000:00010000:3.0:1789760790.715899:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80121856 -> 80125952 00002000:00010000:3.0:1789760790.715907:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156512 -> 156520 00010000:00010000:3.0:1789760790.715921:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.715946:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.715956:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.715963:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.715967:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.715977:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.716004:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.716013:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1ee8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80121856->80125951] (req 80121856->80125951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.716118:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503893376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11280us (19582us total) trans 4294986868 rc 0/0 00000100:00100000:3.0:1789760790.716130:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19572 00000100:00100000:3.0:1789760790.716140:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720810667213 time 720829427467 state 0x2 ref 0 00000100:00100000:0.0:1789760790.721482:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.721691:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503893504 00000100:00100000:2.0:1789760790.721701:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503893504 00000100:00100000:2.0:1789760790.721754:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720810667213 time 720829427467 state 0x0 ref 1 00000100:00100000:2.0:1789760790.721769:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720829687883 time 720829427467 state 0x1 ref 1 00000100:00100000:2.0:1789760790.721799:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720829687883 time 720846405602 state 0x0 ref 1 00000100:00100000:2.0:1789760790.721811:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503893504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.721857:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80125952, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26368 00010000:00010000:2.0:1789760790.721867:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.721883:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.721888:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.721894:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.721907:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.728794:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986869 is committed 00000100:00100000:3.0:1789760790.738579:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5e01be3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523614720:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760790.738649:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760790.738742:0:8432:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614720 00000100:00100000:3.0:1789760790.738750:0:8432:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614720 00000100:00100000:3.0:1789760790.738802:0:8432:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1789760790.738812:0:8432:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523614720:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760790.738834:0:8432:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3843883008 avail 3608891392 00000100:00100000:3.0:1789760790.738877:0:8432:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523614720:12345-0@lo:13:osp-pre-0-1.0 Request processed in 67us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760790.738884:0:8432:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1789760790.738934:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5e01be3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523614720:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760790.739163:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc687b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523614848:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760790.739194:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:2.0:1789760790.740362:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80125952 -> 80130048 00002000:00010000:2.0:1789760790.740369:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156520 -> 156528 00010000:00010000:2.0:1789760790.740381:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.740410:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.740424:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.740431:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.740436:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.740445:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.740465:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.740472:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1eef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80125952->80130047] (req 80125952->80130047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.740562:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503893504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18750us (19087us total) trans 4294986869 rc 0/0 00000100:00100000:2.0:1789760790.740572:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19573 00000100:00100000:2.0:1789760790.740581:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720829687883 time 720846405602 state 0x2 ref 0 00000100:00100000:0.0:1789760790.741385:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614848 00000100:00100000:0.0:1789760790.741405:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614848 00000100:00100000:0.0:1789760790.741452:0:15166:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1789760790.741462:0:15166:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523614848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760790.741485:0:15166:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3843883008 avail 3608891392 00000100:00100000:0.0:1789760790.747733:0:15166:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523614848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 6261us (8527us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760790.747749:0:15166:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:3.0:1789760790.747758:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760790.747812:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503893632 00000100:00100000:0.0:1789760790.747821:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503893632 00000100:00100000:0.0:1789760790.747868:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720829687883 time 720846405602 state 0x0 ref 1 00000100:00100000:0.0:1789760790.747883:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720846666018 time 720846405602 state 0x1 ref 1 00000100:00100000:0.0:1789760790.747912:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720846666018 time 720872518578 state 0x0 ref 1 00000100:00100000:0.0:1789760790.747924:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503893632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.747971:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80130048, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26370 00010000:00010000:0.0:1789760790.747981:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.748002:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760790.748007:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.748013:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.748027:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.749947:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc687b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523614848:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760790.756380:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986870 is committed 00002000:00010000:0.0:1789760790.756569:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80130048 -> 80134144 00002000:00010000:0.0:1789760790.756575:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156528 -> 156536 00010000:00010000:0.0:1789760790.756587:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.756605:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.756613:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.756620:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.756624:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.756634:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.756655:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.756662:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1ef6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80130048->80134143] (req 80130048->80134143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.756759:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503893632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8832us (9000us total) trans 4294986870 rc 0/0 00000100:00100000:0.0:1789760790.756769:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19574 00000100:00100000:0.0:1789760790.756778:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720846666018 time 720872518578 state 0x2 ref 0 00000100:00100000:1.0:1789760790.759753:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760790.759997:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503893760 00000100:00100000:0.0:1789760790.760006:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503893760 00000100:00100000:0.0:1789760790.760060:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720846666018 time 720872518578 state 0x0 ref 1 00000100:00100000:0.0:1789760790.760074:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720872778994 time 720872518578 state 0x1 ref 1 00000100:00100000:0.0:1789760790.760102:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720872778994 time 720884709233 state 0x0 ref 1 00000100:00100000:0.0:1789760790.760114:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503893760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.760164:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80134144, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26378 00010000:00010000:0.0:1789760790.760173:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.760189:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760790.760193:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.760199:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.760213:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.765991:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986871 is committed 00002000:00010000:0.0:1789760790.766091:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80134144 -> 80138240 00002000:00010000:0.0:1789760790.766097:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156536 -> 156544 00010000:00010000:0.0:1789760790.766109:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.766120:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.766134:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.766141:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.766146:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.766155:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.766173:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.766181:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1efd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80134144->80138239] (req 80134144->80138239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.766335:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503893760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6216us (6586us total) trans 4294986871 rc 0/0 00000100:00100000:0.0:1789760790.766345:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19575 00000100:00100000:0.0:1789760790.766354:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720872778994 time 720884709233 state 0x2 ref 0 00000100:00100000:3.0:1789760790.770115:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.770345:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503893888 00000100:00100000:1.0:1789760790.770355:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503893888 00000100:00100000:1.0:1789760790.770478:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720872778994 time 720884709233 state 0x0 ref 1 00000100:00100000:1.0:1789760790.770492:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720884969649 time 720884709233 state 0x1 ref 1 00000100:00100000:1.0:1789760790.770563:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720884969649 time 720895169259 state 0x0 ref 1 00000100:00100000:1.0:1789760790.770576:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503893888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.770622:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80138240, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26380 00010000:00010000:1.0:1789760790.770632:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.770648:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.770653:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.770659:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.770672:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.775464:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986872 is committed 00002000:00010000:3.0:1789760790.775663:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80138240 -> 80142336 00002000:00010000:3.0:1789760790.775668:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156544 -> 156552 00010000:00010000:3.0:1789760790.775681:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.775694:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.775702:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.775709:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.775713:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.775722:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.775742:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.775749:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f1f04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80138240->80142335] (req 80138240->80142335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.775834:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503893888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5255us (5723us total) trans 4294986872 rc 0/0 00000100:00100000:3.0:1789760790.775847:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19576 00000100:00100000:3.0:1789760790.775857:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720884969649 time 720895169259 state 0x2 ref 0 00000100:00100000:0.0:1789760790.779009:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.779093:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894016 00000100:00100000:2.0:1789760790.779104:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894016 00000100:00100000:2.0:1789760790.779158:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720884969649 time 720895169259 state 0x0 ref 1 00000100:00100000:2.0:1789760790.779173:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720895429675 time 720895169259 state 0x1 ref 1 00000100:00100000:2.0:1789760790.779202:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720895429675 time 720903808648 state 0x0 ref 1 00000100:00100000:2.0:1789760790.779214:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503894016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.779267:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80142336, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x26388 00010000:00010000:2.0:1789760790.779276:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.779292:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.779296:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.779303:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.779317:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.788646:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986873 is committed 00002000:00010000:0.0:1789760790.790614:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80142336 -> 80146432 00002000:00010000:0.0:1789760790.790622:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:0.0:1789760790.790625:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00002000:00010000:0.0:1789760790.790627:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156552 -> 156560 00010000:00010000:0.0:1789760790.790642:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.790655:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.790663:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.790671:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.790676:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.790686:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.790706:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.790715:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80142336->80146431] (req 80142336->80146431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.790820:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503894016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11601us (11815us total) trans 4294986873 rc 0/0 00000100:00100000:0.0:1789760790.790831:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19577 00000100:00100000:0.0:1789760790.790842:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720895429675 time 720903808648 state 0x2 ref 0 00000100:00100000:1.0:1789760790.794149:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760790.794230:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894144 00000100:00100000:0.0:1789760790.794249:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894144 00000100:00100000:0.0:1789760790.794307:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720895429675 time 720903808648 state 0x0 ref 1 00000100:00100000:0.0:1789760790.794321:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720904069064 time 720903808648 state 0x1 ref 1 00000100:00100000:0.0:1789760790.794351:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720904069064 time 720918957869 state 0x0 ref 1 00000100:00100000:0.0:1789760790.794363:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.794436:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80146432, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26390 00010000:00010000:0.0:1789760790.794445:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.794460:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760790.794465:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.794471:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.794484:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.798836:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986874 is committed 00002000:00010000:0.0:1789760790.798984:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80146432 -> 80150528 00002000:00010000:0.0:1789760790.798990:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156560 -> 156568 00010000:00010000:0.0:1789760790.799004:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.799024:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.799034:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.799041:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.799045:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.799055:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.799073:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.799081:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80146432->80150527] (req 80146432->80150527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.799257:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4890us (5114us total) trans 4294986874 rc 0/0 00000100:00100000:0.0:1789760790.799269:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19578 00000100:00100000:0.0:1789760790.799279:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720904069064 time 720918957869 state 0x2 ref 0 00000100:00100000:0.0:1789760790.806053:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.806209:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894272 00000100:00100000:2.0:1789760790.806219:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894272 00000100:00100000:2.0:1789760790.806334:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720904069064 time 720918957869 state 0x0 ref 1 00000100:00100000:2.0:1789760790.806349:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720919218285 time 720918957869 state 0x1 ref 1 00000100:00100000:2.0:1789760790.806419:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720919218285 time 720931025639 state 0x0 ref 1 00000100:00100000:2.0:1789760790.806432:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.806478:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80150528, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26398 00010000:00010000:2.0:1789760790.806487:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.806513:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.806518:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.806525:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.806545:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.811146:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986875 is committed 00002000:00010000:2.0:1789760790.811302:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80150528 -> 80154624 00002000:00010000:2.0:1789760790.811308:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156568 -> 156576 00010000:00010000:2.0:1789760790.811322:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.811335:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.811344:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.811353:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.811358:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.811367:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.811636:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.811648:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80150528->80154623] (req 80150528->80154623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.811787:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5350us (5737us total) trans 4294986875 rc 0/0 00000100:00100000:2.0:1789760790.811799:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19579 00000100:00100000:2.0:1789760790.811808:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720919218285 time 720931025639 state 0x2 ref 0 00000100:00100000:1.0:1789760790.823565:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.823764:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894400 00000100:00100000:3.0:1789760790.823773:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894400 00000100:00100000:3.0:1789760790.823828:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720919218285 time 720931025639 state 0x0 ref 1 00000100:00100000:3.0:1789760790.823843:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720931286055 time 720931025639 state 0x1 ref 1 00000100:00100000:0.0:1789760790.823999:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720931286055 time 720948603557 state 0x0 ref 1 00000100:00100000:0.0:1789760790.824015:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.824073:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80154624, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263a0 00010000:00010000:0.0:1789760790.824084:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.824099:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760790.824103:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.824110:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.824125:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.828077:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986876 is committed 00002000:00010000:0.0:1789760790.828190:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80154624 -> 80158720 00002000:00010000:0.0:1789760790.828195:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156576 -> 156584 00010000:00010000:0.0:1789760790.828208:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.828275:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.828286:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.828294:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.828299:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.828309:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.828330:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.828339:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80154624->80158719] (req 80154624->80158719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.828756:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4736us (5193us total) trans 4294986876 rc 0/0 00000100:00100000:0.0:1789760790.828767:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19580 00000100:00100000:0.0:1789760790.828777:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720931286055 time 720948603557 state 0x2 ref 0 00000100:00100000:0.0:1789760790.833968:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.834056:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894528 00000100:00100000:2.0:1789760790.834066:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894528 00000100:00100000:2.0:1789760790.834121:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720931286055 time 720948603557 state 0x0 ref 1 00000100:00100000:2.0:1789760790.834135:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720948863973 time 720948603557 state 0x1 ref 1 00000100:00100000:2.0:1789760790.834166:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720948863973 time 720958772651 state 0x0 ref 1 00000100:00100000:2.0:1789760790.834177:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.834229:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80158720, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263a8 00010000:00010000:2.0:1789760790.834240:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.834255:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.834261:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.834268:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.834282:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.838169:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986877 is committed 00002000:00010000:2.0:1789760790.845439:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80158720 -> 80162816 00002000:00010000:2.0:1789760790.845449:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156584 -> 156592 00010000:00010000:2.0:1789760790.845463:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.845476:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.845486:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.845494:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.845511:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.845521:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.845540:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.845690:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80158720->80162815] (req 80158720->80162815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.845796:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11615us (11832us total) trans 4294986877 rc 0/0 00000100:00100000:2.0:1789760790.845807:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19581 00000100:00100000:2.0:1789760790.845818:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720948863973 time 720958772651 state 0x2 ref 0 00000100:00100000:3.0:1789760790.848832:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.848991:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894656 00000100:00100000:1.0:1789760790.849001:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894656 00000100:00100000:1.0:1789760790.849057:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720948863973 time 720958772651 state 0x0 ref 1 00000100:00100000:1.0:1789760790.849071:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720959033067 time 720958772651 state 0x1 ref 1 00000100:00100000:1.0:1789760790.849103:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720959033067 time 720973709813 state 0x0 ref 1 00000100:00100000:1.0:1789760790.849116:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.849162:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80162816, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263b0 00010000:00010000:1.0:1789760790.849172:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.849186:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760790.849191:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.849197:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.849210:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.853125:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986878 is committed 00002000:00010000:1.0:1789760790.853252:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80162816 -> 80166912 00002000:00010000:1.0:1789760790.853258:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156592 -> 156600 00010000:00010000:1.0:1789760790.853270:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.853288:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.853298:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.853306:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.853311:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.853321:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.853339:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.853348:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f1f2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80162816->80166911] (req 80162816->80166911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.853479:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4359us (4651us total) trans 4294986878 rc 0/0 00000100:00100000:1.0:1789760790.853492:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19582 00000100:00100000:1.0:1789760790.853512:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720959033067 time 720973709813 state 0x2 ref 0 00000100:00100000:1.0:1789760790.857299:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760790.861964:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894784 00000100:00100000:0.0:1789760790.861979:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894784 00000100:00100000:0.0:1789760790.862040:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720959033067 time 720973709813 state 0x0 ref 1 00000100:00100000:0.0:1789760790.862056:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720973970229 time 720973709813 state 0x1 ref 1 00000100:00100000:0.0:1789760790.862141:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720973970229 time 720986747138 state 0x0 ref 1 00000100:00100000:0.0:1789760790.862156:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503894784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.862206:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80166912, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263b8 00010000:00010000:0.0:1789760790.862216:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.862242:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760790.862246:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.862253:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.862267:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.867067:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986879 is committed 00002000:00010000:0.0:1789760790.867349:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80166912 -> 80171008 00002000:00010000:0.0:1789760790.867354:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156600 -> 156608 00010000:00010000:0.0:1789760790.867367:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.867379:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.867488:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.867508:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.867513:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.867530:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.867653:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.867662:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f1f35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80166912->80171007] (req 80166912->80171007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.867743:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503894784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5585us (10448us total) trans 4294986879 rc 0/0 00000100:00100000:0.0:1789760790.867753:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19583 00000100:00100000:0.0:1789760790.867762:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720973970229 time 720986747138 state 0x2 ref 0 00000100:00100000:3.0:1789760790.870881:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.871170:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503894912 00000100:00100000:2.0:1789760790.871180:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503894912 00000100:00100000:2.0:1789760790.871248:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720973970229 time 720986747138 state 0x0 ref 1 00000100:00100000:2.0:1789760790.871264:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720987007554 time 720986747138 state 0x1 ref 1 00000100:00100000:2.0:1789760790.871293:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720987007554 time 720995900201 state 0x0 ref 1 00000100:00100000:2.0:1789760790.871306:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.871352:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80171008, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263c0 00010000:00010000:2.0:1789760790.871362:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.871379:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760790.871384:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.871411:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.871434:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.882516:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986880 is committed 00002000:00010000:2.0:1789760790.882694:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80171008 -> 80175104 00002000:00010000:2.0:1789760790.882700:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156608 -> 156616 00010000:00010000:2.0:1789760790.882716:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.882728:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.882738:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.882745:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.882749:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.882758:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.882779:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.882787:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80171008->80175103] (req 80171008->80175103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.882891:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503894912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11582us (12026us total) trans 4294986880 rc 0/0 00000100:00100000:2.0:1789760790.882902:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19584 00000100:00100000:2.0:1789760790.882913:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720987007554 time 720995900201 state 0x2 ref 0 00000100:00100000:1.0:1789760790.890356:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.890661:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895040 00000100:00100000:3.0:1789760790.890672:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895040 00000100:00100000:3.0:1789760790.890729:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720987007554 time 720995900201 state 0x0 ref 1 00000100:00100000:3.0:1789760790.890744:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720996160617 time 720995900201 state 0x1 ref 1 00000100:00100000:3.0:1789760790.890809:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720996160617 time 721015414805 state 0x0 ref 1 00000100:00100000:3.0:1789760790.890823:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.890870:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80175104, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263c8 00010000:00010000:3.0:1789760790.890879:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.890893:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760790.890898:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.890905:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.890920:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.905277:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986881 is committed 00002000:00010000:3.0:1789760790.905450:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80175104 -> 80179200 00002000:00010000:3.0:1789760790.905455:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156616 -> 156624 00010000:00010000:3.0:1789760790.905468:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.905480:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.905489:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.905496:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760790.905509:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.905524:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.905551:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.905559:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80175104->80179199] (req 80175104->80179199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.905699:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14873us (15348us total) trans 4294986881 rc 0/0 00000100:00100000:3.0:1789760790.905709:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19585 00000100:00100000:3.0:1789760790.905718:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720996160617 time 721015414805 state 0x2 ref 0 00000100:00100000:1.0:1789760790.916756:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760790.916857:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895168 00000100:00100000:0.0:1789760790.916867:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895168 00000100:00100000:0.0:1789760790.916921:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 720996160617 time 721015414805 state 0x0 ref 1 00000100:00100000:0.0:1789760790.917048:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721015675221 time 721015414805 state 0x1 ref 1 00000100:00100000:0.0:1789760790.917080:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721015675221 time 721041687083 state 0x0 ref 1 00000100:00100000:0.0:1789760790.917093:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760790.917137:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80179200, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263d0 00010000:00010000:0.0:1789760790.917147:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.917162:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760790.917167:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.917173:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.917186:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.928362:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986882 is committed 00002000:00010000:0.0:1789760790.928512:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80179200 -> 80183296 00002000:00010000:0.0:1789760790.928517:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156624 -> 156632 00010000:00010000:0.0:1789760790.928530:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.928551:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.928559:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.928567:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760790.928571:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.928585:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.928610:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.928618:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f1f4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80179200->80183295] (req 80179200->80183295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.928704:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11608us (11953us total) trans 4294986882 rc 0/0 00000100:00100000:0.0:1789760790.928715:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19586 00000100:00100000:0.0:1789760790.928725:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721015675221 time 721041687083 state 0x2 ref 0 00000100:00100000:0.0:1789760790.932478:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.932999:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895296 00000100:00100000:1.0:1789760790.933010:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895296 00000100:00100000:1.0:1789760790.933064:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721015675221 time 721041687083 state 0x0 ref 1 00000100:00100000:1.0:1789760790.933080:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721041947499 time 721041687083 state 0x1 ref 1 00000100:00100000:1.0:1789760790.933111:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721041947499 time 721057717412 state 0x0 ref 1 00000100:00100000:1.0:1789760790.933124:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.933172:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80183296, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263d8 00010000:00010000:1.0:1789760790.933181:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.933196:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760790.933201:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.933207:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.933219:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.942488:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986883 is committed 00002000:00010000:1.0:1789760790.942662:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80183296 -> 80187392 00002000:00010000:1.0:1789760790.942666:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156632 -> 156640 00010000:00010000:1.0:1789760790.942680:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942691:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942699:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942707:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.942712:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942721:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942739:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942748:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80183296->80187391] (req 80183296->80187391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.942967:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9839us (10494us total) trans 4294986883 rc 0/0 00000100:00100000:1.0:1789760790.942977:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19587 00000100:00100000:1.0:1789760790.942987:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721041947499 time 721057717412 state 0x2 ref 0 00000100:00100000:1.0:1789760790.946508:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.946659:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895424 00000100:00100000:2.0:1789760790.946669:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895424 00000100:00100000:2.0:1789760790.946722:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721041947499 time 721057717412 state 0x0 ref 1 00000100:00100000:2.0:1789760790.946739:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721057977828 time 721057717412 state 0x1 ref 1 00000100:00100000:2.0:1789760790.946761:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721057977828 time 721071367776 state 0x0 ref 1 00000100:00100000:2.0:1789760790.946774:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.946834:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80187392, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263e0 00010000:00010000:2.0:1789760790.946843:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.946858:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760790.946863:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.946869:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.946886:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.953385:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986884 is committed 00002000:00010000:2.0:1789760790.953529:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80187392 -> 80191488 00002000:00010000:2.0:1789760790.953535:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156640 -> 156648 00010000:00010000:2.0:1789760790.953559:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.953571:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.953582:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.953589:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.953594:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.953604:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.953623:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.953630:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f1f58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80187392->80191487] (req 80187392->80191487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.953724:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6948us (7232us total) trans 4294986884 rc 0/0 00000100:00100000:2.0:1789760790.953734:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19588 00000100:00100000:2.0:1789760790.953744:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721057977828 time 721071367776 state 0x2 ref 0 00000100:00100000:0.0:1789760790.963089:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760790.963519:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895552 00000100:00100000:3.0:1789760790.963529:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895552 00000100:00100000:3.0:1789760790.963593:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721057977828 time 721071367776 state 0x0 ref 1 00000100:00100000:3.0:1789760790.963608:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721071628192 time 721071367776 state 0x1 ref 1 00000100:00100000:3.0:1789760790.963640:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721071628192 time 721088246556 state 0x0 ref 1 00000100:00100000:3.0:1789760790.963653:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760790.963706:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80191488, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263e8 00010000:00010000:3.0:1789760790.963716:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.963730:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760790.963735:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.963741:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.963756:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.971126:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986885 is committed 00002000:00010000:1.0:1789760790.971494:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80191488 -> 80195584 00002000:00010000:1.0:1789760790.971510:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156648 -> 156656 00010000:00010000:1.0:1789760790.971523:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.971607:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.971619:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.971626:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760790.971631:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.971641:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.971662:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.971670:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f1f5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80191488->80195583] (req 80191488->80195583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.971808:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8151us (8722us total) trans 4294986885 rc 0/0 00000100:00100000:1.0:1789760790.973753:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19589 00000100:00100000:1.0:1789760790.973766:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721071628192 time 721088246556 state 0x2 ref 0 00000100:00100000:0.0:1789760790.975619:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760790.975836:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895680 00000100:00100000:2.0:1789760790.975847:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895680 00000100:00100000:2.0:1789760790.975900:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721071628192 time 721088246556 state 0x0 ref 1 00000100:00100000:2.0:1789760790.975914:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721088506972 time 721088246556 state 0x1 ref 1 00000100:00100000:2.0:1789760790.975943:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721088506972 time 721100549886 state 0x0 ref 1 00000100:00100000:2.0:1789760790.975956:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760790.976005:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80195584, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263f0 00010000:00010000:2.0:1789760790.976015:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.976029:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760790.976034:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.976039:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.976055:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.983793:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986886 is committed 00002000:00010000:2.0:1789760790.984054:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80195584 -> 80199680 00002000:00010000:2.0:1789760790.984059:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156656 -> 156664 00010000:00010000:2.0:1789760790.984072:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.984084:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.984093:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.984101:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760790.984105:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.984115:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.984134:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.984143:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80195584->80199679] (req 80195584->80199679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.984324:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8365us (8708us total) trans 4294986886 rc 0/0 00000100:00100000:2.0:1789760790.984335:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19590 00000100:00100000:2.0:1789760790.984344:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721088506972 time 721100549886 state 0x2 ref 0 00000100:00100000:2.0:1789760790.996023:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760790.996113:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895808 00000100:00100000:1.0:1789760790.996124:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895808 00000100:00100000:1.0:1789760790.996188:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721088506972 time 721100549886 state 0x0 ref 1 00000100:00100000:1.0:1789760790.996204:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721100810302 time 721100549886 state 0x1 ref 1 00000100:00100000:1.0:1789760790.996241:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721100810302 time 721120847187 state 0x0 ref 1 00000100:00100000:1.0:1789760790.996254:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760790.996300:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80199680, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x263f8 00010000:00010000:1.0:1789760790.996309:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.996323:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760790.996328:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.996335:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.996349:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.001019:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986887 is committed 00002000:00010000:1.0:1789760791.001126:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80199680 -> 80203776 00002000:00010000:1.0:1789760791.001131:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156664 -> 156672 00010000:00010000:1.0:1789760791.001144:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.001166:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.001175:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.001182:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.001186:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.001195:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.001213:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.001228:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f1f6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80199680->80203775] (req 80199680->80203775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.001317:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503895808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5060us (5300us total) trans 4294986887 rc 0/0 00000100:00100000:1.0:1789760791.001327:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19591 00000100:00100000:1.0:1789760791.001337:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721100810302 time 721120847187 state 0x2 ref 0 00000100:00100000:0.0:1789760791.010709:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.010820:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503895936 00000100:00100000:2.0:1789760791.010830:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503895936 00000100:00100000:2.0:1789760791.010885:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721100810302 time 721120847187 state 0x0 ref 1 00000100:00100000:2.0:1789760791.010900:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721121107603 time 721120847187 state 0x1 ref 1 00000100:00100000:2.0:1789760791.010929:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721121107603 time 721135535815 state 0x0 ref 1 00000100:00100000:2.0:1789760791.010941:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.010989:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80203776, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26400 00010000:00010000:2.0:1789760791.010997:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.011012:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.011016:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.011021:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.011035:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.016678:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986888 is committed 00002000:00010000:2.0:1789760791.016800:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80203776 -> 80207872 00002000:00010000:2.0:1789760791.016806:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156672 -> 156680 00010000:00010000:2.0:1789760791.016820:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.016833:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.016841:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.016848:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.016853:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.016862:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.016881:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.016890:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f1f74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80203776->80207871] (req 80203776->80207871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.016981:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503895936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6038us (6277us total) trans 4294986888 rc 0/0 00000100:00100000:2.0:1789760791.016991:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19592 00000100:00100000:2.0:1789760791.017002:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721121107603 time 721135535815 state 0x2 ref 0 00000100:00100000:0.0:1789760791.020065:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.020147:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896064 00000100:00100000:3.0:1789760791.020170:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896064 00000100:00100000:3.0:1789760791.020232:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721121107603 time 721135535815 state 0x0 ref 1 00000100:00100000:3.0:1789760791.020247:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721135796231 time 721135535815 state 0x1 ref 1 00000100:00100000:3.0:1789760791.020316:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721135796231 time 721144921782 state 0x0 ref 1 00000100:00100000:3.0:1789760791.020328:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503896064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.020372:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80207872, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26408 00010000:00010000:3.0:1789760791.020381:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.020436:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.020441:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.020448:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.020462:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.026132:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986889 is committed 00002000:00010000:3.0:1789760791.026336:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80207872 -> 80211968 00002000:00010000:3.0:1789760791.026341:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156680 -> 156688 00010000:00010000:3.0:1789760791.026354:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.026366:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.026375:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.026383:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.026410:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.026420:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.026439:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.026452:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80207872->80211967] (req 80207872->80211967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.026663:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503896064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6331us (6601us total) trans 4294986889 rc 0/0 00000100:00100000:3.0:1789760791.026674:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19593 00000100:00100000:3.0:1789760791.026683:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721135796231 time 721144921782 state 0x2 ref 0 00000100:00100000:1.0:1789760791.030673:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.030765:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896192 00000100:00100000:0.0:1789760791.030776:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896192 00000100:00100000:0.0:1789760791.030829:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721135796231 time 721144921782 state 0x0 ref 1 00000100:00100000:0.0:1789760791.030844:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721145182198 time 721144921782 state 0x1 ref 1 00000100:00100000:0.0:1789760791.030882:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721145182198 time 721155488409 state 0x0 ref 1 00000100:00100000:0.0:1789760791.030896:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503896192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.030943:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80211968, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26410 00010000:00010000:0.0:1789760791.030952:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.030966:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.030970:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.030976:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.030989:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.036758:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986890 is committed 00002000:00010000:0.0:1789760791.036985:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80211968 -> 80216064 00002000:00010000:0.0:1789760791.036991:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156688 -> 156696 00010000:00010000:0.0:1789760791.037004:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.037015:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.037023:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.037036:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.037041:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.037049:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.037067:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.037076:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80211968->80216063] (req 80211968->80216063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.037179:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503896192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6281us (6511us total) trans 4294986890 rc 0/0 00000100:00100000:0.0:1789760791.037189:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19594 00000100:00100000:0.0:1789760791.037198:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721145182198 time 721155488409 state 0x2 ref 0 00000100:00100000:0.0:1789760791.042438:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.042528:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896320 00000100:00100000:1.0:1789760791.042548:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896320 00000100:00100000:1.0:1789760791.042603:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721145182198 time 721155488409 state 0x0 ref 1 00000100:00100000:1.0:1789760791.042620:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721155748825 time 721155488409 state 0x1 ref 1 00000100:00100000:1.0:1789760791.042650:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721155748825 time 721167257129 state 0x0 ref 1 00000100:00100000:1.0:1789760791.042663:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503896320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.042708:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80216064, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26418 00010000:00010000:1.0:1789760791.042717:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.042733:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.042738:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.042745:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.042758:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.049107:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986891 is committed 00002000:00010000:1.0:1789760791.049554:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80216064 -> 80220160 00002000:00010000:1.0:1789760791.049559:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156696 -> 156704 00010000:00010000:1.0:1789760791.049572:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.049590:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.049599:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.049606:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.049610:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.049618:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.049636:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.049645:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f1f89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80216064->80220159] (req 80216064->80220159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.049742:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503896320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7075us (7308us total) trans 4294986891 rc 0/0 00000100:00100000:1.0:1789760791.049753:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19595 00000100:00100000:1.0:1789760791.049764:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721155748825 time 721167257129 state 0x2 ref 0 00000100:00100000:3.0:1789760791.052490:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.054349:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896448 00000100:00100000:2.0:1789760791.054363:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896448 00000100:00100000:2.0:1789760791.054521:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721155748825 time 721167257129 state 0x0 ref 1 00000100:00100000:2.0:1789760791.054535:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721167517545 time 721167257129 state 0x1 ref 1 00000100:00100000:2.0:1789760791.054630:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721167517545 time 721179235838 state 0x0 ref 1 00000100:00100000:2.0:1789760791.054645:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.054700:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80220160, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26420 00010000:00010000:2.0:1789760791.054710:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.054726:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.054730:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.054736:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.054749:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.064223:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760791.066934:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986892 is committed 00002000:00010000:3.0:1789760791.067340:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80220160 -> 80224256 00002000:00010000:3.0:1789760791.067346:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156704 -> 156712 00010000:00010000:3.0:1789760791.067359:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.067372:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.067381:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.067388:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.067394:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.067411:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.067432:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.067441:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f1f90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80220160->80224255] (req 80220160->80224255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.068728:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14076us (16237us total) trans 4294986892 rc 0/0 00000100:00100000:3.0:1789760791.068742:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19596 00000100:00100000:3.0:1789760791.068752:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721167517545 time 721179235838 state 0x2 ref 0 00000100:00100000:2.0:1789760791.075886:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.076252:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896576 00000100:00100000:3.0:1789760791.076262:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896576 00000100:00100000:3.0:1789760791.076319:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721167517545 time 721179235838 state 0x0 ref 1 00000100:00100000:3.0:1789760791.076334:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721179496254 time 721179235838 state 0x1 ref 1 00000100:00100000:3.0:1789760791.076449:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721179496254 time 721201055040 state 0x0 ref 1 00000100:00100000:3.0:1789760791.076464:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.076524:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80224256, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26428 00010000:00010000:3.0:1789760791.076535:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.076551:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.076556:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.076562:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.076576:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.089256:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986893 is committed 00002000:00010000:1.0:1789760791.089561:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80224256 -> 80228352 00002000:00010000:1.0:1789760791.089567:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156712 -> 156720 00010000:00010000:1.0:1789760791.089580:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.089592:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.089599:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.089606:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.089610:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.089618:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.089636:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.089642:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1f97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80224256->80228351] (req 80224256->80228351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.089746:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13280us (13865us total) trans 4294986893 rc 0/0 00000100:00100000:1.0:1789760791.089756:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19597 00000100:00100000:1.0:1789760791.090289:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721179496254 time 721201055040 state 0x2 ref 0 00000100:00100000:1.0:1789760791.095100:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.095249:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896704 00000100:00100000:0.0:1789760791.095259:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896704 00000100:00100000:0.0:1789760791.095312:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721179496254 time 721201055040 state 0x0 ref 1 00000100:00100000:0.0:1789760791.095325:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721201315456 time 721201055040 state 0x1 ref 1 00000100:00100000:0.0:1789760791.095429:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721201315456 time 721219995590 state 0x0 ref 1 00000100:00100000:0.0:1789760791.095442:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.095487:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80228352, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26430 00010000:00010000:0.0:1789760791.095495:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.095520:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.095524:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.095529:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.095542:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.103164:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986894 is committed 00002000:00010000:0.0:1789760791.104003:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80228352 -> 80232448 00002000:00010000:0.0:1789760791.104009:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156720 -> 156728 00010000:00010000:0.0:1789760791.104022:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.104033:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.104041:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.104048:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.104053:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.104061:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.104078:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.104085:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f1f9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80228352->80232447] (req 80228352->80232447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.104175:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8730us (9080us total) trans 4294986894 rc 0/0 00000100:00100000:0.0:1789760791.104185:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19598 00000100:00100000:0.0:1789760791.104194:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721201315456 time 721219995590 state 0x2 ref 0 00000100:00100000:3.0:1789760791.113704:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.113768:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896832 00000100:00100000:1.0:1789760791.113779:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896832 00000100:00100000:1.0:1789760791.114023:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721201315456 time 721219995590 state 0x0 ref 1 00000100:00100000:1.0:1789760791.114049:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721220256006 time 721219995590 state 0x1 ref 1 00000100:00100000:1.0:1789760791.114093:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721220256006 time 721238699827 state 0x0 ref 1 00000100:00100000:1.0:1789760791.114107:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503896832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.114184:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80232448, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26438 00010000:00010000:1.0:1789760791.114193:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.114209:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.114213:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.114218:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.114261:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.123143:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986895 is committed 00002000:00010000:1.0:1789760791.123382:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80232448 -> 80236544 00002000:00010000:1.0:1789760791.123624:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156728 -> 156736 00010000:00010000:1.0:1789760791.123639:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.123656:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.123664:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.123670:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.123675:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.123682:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.123700:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.123707:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f1fa5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80232448->80236543] (req 80232448->80236543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.123828:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503896832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9720us (10131us total) trans 4294986895 rc 0/0 00000100:00100000:1.0:1789760791.123838:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19599 00000100:00100000:1.0:1789760791.123855:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721220256006 time 721238699827 state 0x2 ref 0 00000100:00100000:2.0:1789760791.136341:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.136530:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503896960 00000100:00100000:3.0:1789760791.136551:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503896960 00000100:00100000:3.0:1789760791.136608:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721220256006 time 721238699827 state 0x0 ref 1 00000100:00100000:3.0:1789760791.136624:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721238960243 time 721238699827 state 0x1 ref 1 00000100:00100000:3.0:1789760791.136656:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721238960243 time 721261262083 state 0x0 ref 1 00000100:00100000:3.0:1789760791.136668:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.136719:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80236544, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26440 00010000:00010000:3.0:1789760791.136728:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.136745:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.136751:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.136759:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.136784:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.142833:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986896 is committed 00002000:00010000:3.0:1789760791.143012:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80236544 -> 80240640 00002000:00010000:3.0:1789760791.143017:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156736 -> 156744 00010000:00010000:3.0:1789760791.143030:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.143042:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.143052:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.143060:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.143064:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.143074:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.143093:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.143106:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80236544->80240639] (req 80236544->80240639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.143337:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503896960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6665us (6999us total) trans 4294986896 rc 0/0 00000100:00100000:3.0:1789760791.143348:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19600 00000100:00100000:3.0:1789760791.143358:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721238960243 time 721261262083 state 0x2 ref 0 00000100:00100000:0.0:1789760791.152647:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.152711:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897088 00000100:00100000:2.0:1789760791.152722:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897088 00000100:00100000:2.0:1789760791.152771:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721238960243 time 721261262083 state 0x0 ref 1 00000100:00100000:2.0:1789760791.152787:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721261522499 time 721261262083 state 0x1 ref 1 00000100:00100000:2.0:1789760791.152819:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721261522499 time 721277425909 state 0x0 ref 1 00000100:00100000:2.0:1789760791.152832:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.152881:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80240640, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26448 00010000:00010000:2.0:1789760791.152891:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.152907:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.152912:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.152919:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.152934:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.158771:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986897 is committed 00002000:00010000:1.0:1789760791.159264:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80240640 -> 80244736 00002000:00010000:1.0:1789760791.159362:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156744 -> 156752 00010000:00010000:1.0:1789760791.159378:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.159412:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.159422:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.159430:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.159436:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.159446:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.159464:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.159472:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f1fb3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80240640->80244735] (req 80240640->80244735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.159710:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6875us (7067us total) trans 4294986897 rc 0/0 00000100:00100000:1.0:1789760791.159722:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19601 00000100:00100000:1.0:1789760791.159733:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721261522499 time 721277425909 state 0x2 ref 0 00000100:00100000:2.0:1789760791.174964:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.175213:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897216 00000100:00100000:0.0:1789760791.175321:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897216 00000100:00100000:0.0:1789760791.175377:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721261522499 time 721277425909 state 0x0 ref 1 00000100:00100000:0.0:1789760791.175433:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721277686325 time 721277425909 state 0x1 ref 1 00000100:00100000:0.0:1789760791.175464:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721277686325 time 721300070814 state 0x0 ref 1 00000100:00100000:0.0:1789760791.175477:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.175532:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80244736, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26450 00010000:00010000:0.0:1789760791.175888:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175906:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.175910:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.175916:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175930:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.183407:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986898 is committed 00002000:00010000:0.0:1789760791.183492:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80244736 -> 80248832 00002000:00010000:0.0:1789760791.183505:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156752 -> 156760 00010000:00010000:0.0:1789760791.183522:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.183535:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.184622:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.184632:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.184637:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.184653:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.184675:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.184683:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f1fba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80244736->80248831] (req 80244736->80248831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.184772:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9293us (9814us total) trans 4294986898 rc 0/0 00000100:00100000:0.0:1789760791.184783:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19602 00000100:00100000:0.0:1789760791.184793:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721277686325 time 721300070814 state 0x2 ref 0 00000100:00100000:2.0:1789760791.188098:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.188367:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897344 00000100:00100000:1.0:1789760791.188377:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897344 00000100:00100000:1.0:1789760791.188520:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721277686325 time 721300070814 state 0x0 ref 1 00000100:00100000:1.0:1789760791.188536:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721300331230 time 721300070814 state 0x1 ref 1 00000100:00100000:1.0:1789760791.188559:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721300331230 time 721313166246 state 0x0 ref 1 00000100:00100000:1.0:1789760791.188570:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503897344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.188619:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80248832, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26458 00010000:00010000:1.0:1789760791.188629:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.188645:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.188649:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.188655:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.188668:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.198203:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986899 is committed 00002000:00010000:1.0:1789760791.198530:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80248832 -> 80252928 00002000:00010000:1.0:1789760791.198535:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156760 -> 156768 00010000:00010000:1.0:1789760791.198558:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198577:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198586:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198592:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.198597:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198604:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198624:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198631:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80248832->80252927] (req 80248832->80252927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.198732:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503897344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10158us (10637us total) trans 4294986899 rc 0/0 00000100:00100000:1.0:1789760791.198742:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19603 00000100:00100000:1.0:1789760791.198751:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721300331230 time 721313166246 state 0x2 ref 0 00000100:00100000:1.0:1789760791.201594:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.201880:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897472 00000100:00100000:3.0:1789760791.201892:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897472 00000100:00100000:3.0:1789760791.201945:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721300331230 time 721313166246 state 0x0 ref 1 00000100:00100000:3.0:1789760791.201959:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721313426662 time 721313166246 state 0x1 ref 1 00000100:00100000:3.0:1789760791.201988:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721313426662 time 721326594694 state 0x0 ref 1 00000100:00100000:3.0:1789760791.202001:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.202367:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80252928, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26460 00010000:00010000:3.0:1789760791.202377:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.202425:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.202430:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.202437:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.202456:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.209466:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986900 is committed 00002000:00010000:3.0:1789760791.209971:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80252928 -> 80257024 00002000:00010000:3.0:1789760791.209977:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156768 -> 156776 00010000:00010000:3.0:1789760791.209990:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.210021:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.210031:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.210039:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.210044:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.210052:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.210073:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.210081:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f1fc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80252928->80257023] (req 80252928->80257023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.210180:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8177us (8591us total) trans 4294986900 rc 0/0 00000100:00100000:3.0:1789760791.210190:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19604 00000100:00100000:3.0:1789760791.210200:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721313426662 time 721326594694 state 0x2 ref 0 00000100:00100000:2.0:1789760791.213996:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.214431:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897600 00000100:00100000:1.0:1789760791.214453:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897600 00000100:00100000:1.0:1789760791.214519:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721313426662 time 721326594694 state 0x0 ref 1 00000100:00100000:1.0:1789760791.214538:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721326855110 time 721326594694 state 0x1 ref 1 00000100:00100000:1.0:1789760791.214570:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721326855110 time 721339177264 state 0x0 ref 1 00000100:00100000:1.0:1789760791.214583:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503897600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.214635:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80257024, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26468 00010000:00010000:1.0:1789760791.214645:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.214666:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.214670:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.214676:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.214689:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.220514:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986901 is committed 00002000:00010000:1.0:1789760791.220622:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80257024 -> 80261120 00002000:00010000:1.0:1789760791.220627:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156776 -> 156784 00010000:00010000:1.0:1789760791.220640:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.220654:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.220664:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.220671:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.220676:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.220684:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.220703:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.220710:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1fcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80257024->80261119] (req 80257024->80261119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.220801:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503897600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6214us (6809us total) trans 4294986901 rc 0/0 00000100:00100000:1.0:1789760791.220810:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19605 00000100:00100000:1.0:1789760791.220820:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721326855110 time 721339177264 state 0x2 ref 0 00000100:00100000:2.0:1789760791.227662:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.227746:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897728 00000100:00100000:1.0:1789760791.227755:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897728 00000100:00100000:1.0:1789760791.227811:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721326855110 time 721339177264 state 0x0 ref 1 00000100:00100000:1.0:1789760791.227827:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721339437680 time 721339177264 state 0x1 ref 1 00000100:00100000:1.0:1789760791.227863:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721339437680 time 721352470036 state 0x0 ref 1 00000100:00100000:1.0:1789760791.227876:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503897728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.227927:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80261120, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26470 00010000:00010000:1.0:1789760791.227936:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.227952:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.227957:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.227963:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.227978:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.234458:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986902 is committed 00002000:00010000:1.0:1789760791.234580:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80261120 -> 80265216 00002000:00010000:1.0:1789760791.234585:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156784 -> 156792 00010000:00010000:1.0:1789760791.234598:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.234610:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.234619:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.234632:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.234636:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.234644:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.234662:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.234669:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80261120->80265215] (req 80261120->80265215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.234763:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503897728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6884us (7106us total) trans 4294986902 rc 0/0 00000100:00100000:1.0:1789760791.234774:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19606 00000100:00100000:1.0:1789760791.234784:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721339437680 time 721352470036 state 0x2 ref 0 00000100:00100000:0.0:1789760791.240373:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.240518:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897856 00000100:00100000:1.0:1789760791.240527:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897856 00000100:00100000:1.0:1789760791.240588:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721339437680 time 721352470036 state 0x0 ref 1 00000100:00100000:1.0:1789760791.240603:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721352730452 time 721352470036 state 0x1 ref 1 00000100:00100000:1.0:1789760791.240624:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721352730452 time 721365231956 state 0x0 ref 1 00000100:00100000:1.0:1789760791.240638:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.240680:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80265216, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26478 00010000:00010000:1.0:1789760791.240690:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.240705:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.240710:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.240716:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.240729:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.249091:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986903 is committed 00002000:00010000:1.0:1789760791.249239:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80265216 -> 80269312 00002000:00010000:1.0:1789760791.249244:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156792 -> 156800 00010000:00010000:1.0:1789760791.249258:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.249271:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.249281:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.249288:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.249293:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.249303:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.249323:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.249331:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f1fdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80265216->80269311] (req 80265216->80269311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.249475:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503897856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8834us (9109us total) trans 4294986903 rc 0/0 00000100:00100000:1.0:1789760791.249486:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19607 00000100:00100000:1.0:1789760791.249507:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721352730452 time 721365231956 state 0x2 ref 0 00000100:00100000:2.0:1789760791.253074:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.253155:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503897984 00000100:00100000:3.0:1789760791.253166:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503897984 00000100:00100000:3.0:1789760791.253232:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721352730452 time 721365231956 state 0x0 ref 1 00000100:00100000:3.0:1789760791.253249:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721365492372 time 721365231956 state 0x1 ref 1 00000100:00100000:3.0:1789760791.253274:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721365492372 time 721377880744 state 0x0 ref 1 00000100:00100000:3.0:1789760791.253286:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503897984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.253331:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80269312, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26480 00010000:00010000:3.0:1789760791.253340:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.253355:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.253360:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.253366:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.253379:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.259698:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986904 is committed 00002000:00010000:0.0:1789760791.260073:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80269312 -> 80273408 00002000:00010000:0.0:1789760791.260079:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156800 -> 156808 00010000:00010000:0.0:1789760791.260093:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260107:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260122:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260130:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.260135:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260144:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260165:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260173:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f1fe4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80269312->80273407] (req 80269312->80273407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.260443:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503897984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7153us (7374us total) trans 4294986904 rc 0/0 00000100:00100000:0.0:1789760791.260455:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19608 00000100:00100000:0.0:1789760791.260466:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721365492372 time 721377880744 state 0x2 ref 0 00000100:00100000:3.0:1789760791.265917:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.266939:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503898112 00000100:00100000:2.0:1789760791.266952:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503898112 00000100:00100000:2.0:1789760791.267010:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721365492372 time 721377880744 state 0x0 ref 1 00000100:00100000:2.0:1789760791.267025:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721378141160 time 721377880744 state 0x1 ref 1 00000100:00100000:2.0:1789760791.267103:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721378141160 time 721391708867 state 0x0 ref 1 00000100:00100000:2.0:1789760791.267117:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.267164:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80273408, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26488 00010000:00010000:2.0:1789760791.267174:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.267190:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.267194:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.267201:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.267215:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.272449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986905 is committed 00002000:00010000:2.0:1789760791.272633:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80273408 -> 80277504 00002000:00010000:2.0:1789760791.272639:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156808 -> 156816 00010000:00010000:2.0:1789760791.272652:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.272665:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.272674:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.272682:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.272686:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.272696:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.272715:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.272723:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f1feb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80273408->80277503] (req 80273408->80277503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.272821:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5702us (6908us total) trans 4294986905 rc 0/0 00000100:00100000:2.0:1789760791.272832:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19609 00000100:00100000:2.0:1789760791.272841:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721378141160 time 721391708867 state 0x2 ref 0 00000100:00100000:1.0:1789760791.275698:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.278297:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503898240 00000100:00100000:0.0:1789760791.278311:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503898240 00000100:00100000:0.0:1789760791.278374:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721378141160 time 721391708867 state 0x0 ref 1 00000100:00100000:0.0:1789760791.278511:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721391969283 time 721391708867 state 0x1 ref 1 00000100:00100000:0.0:1789760791.278698:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721391969283 time 721403303049 state 0x0 ref 1 00000100:00100000:0.0:1789760791.278712:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503898240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.278758:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80277504, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26490 00010000:00010000:0.0:1789760791.278767:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.278783:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.278787:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.278793:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.278807:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.283643:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986906 is committed 00002000:00010000:0.0:1789760791.283754:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80277504 -> 80281600 00002000:00010000:0.0:1789760791.283760:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156816 -> 156824 00010000:00010000:0.0:1789760791.283773:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.283787:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.283795:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.283808:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.283813:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.283836:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.283855:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.283863:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f1ff2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80277504->80281599] (req 80277504->80281599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.283955:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503898240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5241us (8261us total) trans 4294986906 rc 0/0 00000100:00100000:0.0:1789760791.283966:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19610 00000100:00100000:0.0:1789760791.283976:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721391969283 time 721403303049 state 0x2 ref 0 00000100:00100000:0.0:1789760791.287765:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.287994:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503898368 00000100:00100000:1.0:1789760791.288004:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503898368 00000100:00100000:1.0:1789760791.288058:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721391969283 time 721403303049 state 0x0 ref 1 00000100:00100000:1.0:1789760791.288075:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721403563465 time 721403303049 state 0x1 ref 1 00000100:00100000:1.0:1789760791.288159:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721403563465 time 721412764387 state 0x0 ref 1 00000100:00100000:1.0:1789760791.288174:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503898368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.288237:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80281600, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26498 00010000:00010000:1.0:1789760791.288257:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.288272:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.288277:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.288283:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.288299:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.295818:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986907 is committed 00002000:00010000:1.0:1789760791.296030:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80281600 -> 80285696 00002000:00010000:1.0:1789760791.296035:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156824 -> 156832 00010000:00010000:1.0:1789760791.296049:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.296182:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.296191:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.296197:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.296202:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.296216:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.296291:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.296298:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f1ff9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80281600->80285695] (req 80281600->80285695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.296608:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503898368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8432us (8853us total) trans 4294986907 rc 0/0 00000100:00100000:1.0:1789760791.296618:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19611 00000100:00100000:1.0:1789760791.296628:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721403563465 time 721412764387 state 0x2 ref 0 00000100:00100000:3.0:1789760791.300258:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.300646:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503898496 00000100:00100000:0.0:1789760791.300657:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503898496 00000100:00100000:0.0:1789760791.300712:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721403563465 time 721412764387 state 0x0 ref 1 00000100:00100000:0.0:1789760791.300727:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721413024803 time 721412764387 state 0x1 ref 1 00000100:00100000:0.0:1789760791.300805:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721413024803 time 721425409850 state 0x0 ref 1 00000100:00100000:0.0:1789760791.300830:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.300882:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80285696, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264a0 00010000:00010000:0.0:1789760791.300891:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.300908:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.300913:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.300919:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.300932:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.311564:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986908 is committed 00002000:00010000:1.0:1789760791.312718:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80285696 -> 80289792 00002000:00010000:1.0:1789760791.312725:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156832 -> 156840 00010000:00010000:1.0:1789760791.312739:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.312752:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.312762:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.312775:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.312780:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.312790:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.312809:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.312818:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2000 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80285696->80289791] (req 80285696->80289791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.312914:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12092us (12659us total) trans 4294986908 rc 0/0 00000100:00100000:1.0:1789760791.312923:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19612 00000100:00100000:1.0:1789760791.312932:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721413024803 time 721425409850 state 0x2 ref 0 00000100:00100000:0.0:1789760791.319556:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.319640:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503898624 00000100:00100000:2.0:1789760791.319649:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503898624 00000100:00100000:2.0:1789760791.319703:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721413024803 time 721425409850 state 0x0 ref 1 00000100:00100000:2.0:1789760791.319719:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721425670266 time 721425409850 state 0x1 ref 1 00000100:00100000:2.0:1789760791.319746:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721425670266 time 721444352781 state 0x0 ref 1 00000100:00100000:2.0:1789760791.319759:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503898624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.319807:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80289792, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264a8 00010000:00010000:2.0:1789760791.319817:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.319832:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.319837:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.319844:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.319859:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.326054:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986909 is committed 00002000:00010000:2.0:1789760791.326168:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80289792 -> 80293888 00002000:00010000:2.0:1789760791.326173:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156840 -> 156848 00010000:00010000:2.0:1789760791.326185:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.326197:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.326211:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.326275:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.326281:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.326291:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.326311:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.326319:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2007 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80289792->80293887] (req 80289792->80293887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.326470:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503898624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6708us (6919us total) trans 4294986909 rc 0/0 00000100:00100000:2.0:1789760791.326480:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19613 00000100:00100000:2.0:1789760791.326489:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721425670266 time 721444352781 state 0x2 ref 0 00000100:00100000:0.0:1789760791.333190:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.333339:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503898752 00000100:00100000:1.0:1789760791.333350:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503898752 00000100:00100000:1.0:1789760791.333440:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721425670266 time 721444352781 state 0x0 ref 1 00000100:00100000:1.0:1789760791.333455:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721444613197 time 721444352781 state 0x1 ref 1 00000100:00100000:1.0:1789760791.333479:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721444613197 time 721458085732 state 0x0 ref 1 00000100:00100000:1.0:1789760791.333510:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.333560:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80293888, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264b0 00010000:00010000:1.0:1789760791.333570:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333585:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.333590:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.333597:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.333610:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.339268:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986910 is committed 00002000:00010000:1.0:1789760791.339574:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80293888 -> 80297984 00002000:00010000:1.0:1789760791.339579:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156848 -> 156856 00010000:00010000:1.0:1789760791.339592:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.339605:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.339615:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.339622:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.339628:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.339637:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.339655:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.339663:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f200e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80293888->80297983] (req 80293888->80297983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.339761:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6263us (6575us total) trans 4294986910 rc 0/0 00000100:00100000:1.0:1789760791.339771:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19614 00000100:00100000:1.0:1789760791.339781:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721444613197 time 721458085732 state 0x2 ref 0 00000100:00100000:3.0:1789760791.344600:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.344975:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503898880 00000100:00100000:1.0:1789760791.344985:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503898880 00000100:00100000:1.0:1789760791.345041:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721444613197 time 721458085732 state 0x0 ref 1 00000100:00100000:1.0:1789760791.345055:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721458346148 time 721458085732 state 0x1 ref 1 00000100:00100000:1.0:1789760791.345087:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721458346148 time 721469693347 state 0x0 ref 1 00000100:00100000:1.0:1789760791.345109:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.345159:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80297984, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264b8 00010000:00010000:1.0:1789760791.345169:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.345185:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.345189:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.345194:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.345207:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.350180:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986911 is committed 00002000:00010000:1.0:1789760791.350457:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80297984 -> 80302080 00002000:00010000:1.0:1789760791.350462:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156856 -> 156864 00010000:00010000:1.0:1789760791.350474:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.350487:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.350506:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.350514:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.350519:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.350528:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.350548:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.350556:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2015 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80297984->80302079] (req 80297984->80302079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.350644:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503898880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5534us (6049us total) trans 4294986911 rc 0/0 00000100:00100000:1.0:1789760791.350655:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19615 00000100:00100000:1.0:1789760791.350666:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721458346148 time 721469693347 state 0x2 ref 0 00000100:00100000:0.0:1789760791.356738:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.356836:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899008 00000100:00100000:1.0:1789760791.356846:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899008 00000100:00100000:1.0:1789760791.356898:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721458346148 time 721469693347 state 0x0 ref 1 00000100:00100000:1.0:1789760791.356913:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721469953763 time 721469693347 state 0x1 ref 1 00000100:00100000:1.0:1789760791.356952:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721469953763 time 721481558404 state 0x0 ref 1 00000100:00100000:1.0:1789760791.356964:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503899008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.357013:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80302080, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264c0 00010000:00010000:1.0:1789760791.357022:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.357037:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.357043:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.357049:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.357072:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.362301:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986912 is committed 00002000:00010000:1.0:1789760791.362544:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80302080 -> 80306176 00002000:00010000:1.0:1789760791.362549:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156864 -> 156872 00010000:00010000:1.0:1789760791.362563:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.362574:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.362583:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.362591:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.362595:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.362605:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.362624:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.362638:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f201c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80302080->80306175] (req 80302080->80306175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.362815:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503899008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5847us (6081us total) trans 4294986912 rc 0/0 00000100:00100000:1.0:1789760791.362826:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19616 00000100:00100000:1.0:1789760791.362836:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721469953763 time 721481558404 state 0x2 ref 0 00000100:00100000:0.0:1789760791.371027:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.371128:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899136 00000100:00100000:2.0:1789760791.371139:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899136 00000100:00100000:2.0:1789760791.371194:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721469953763 time 721481558404 state 0x0 ref 1 00000100:00100000:2.0:1789760791.371209:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721481818820 time 721481558404 state 0x1 ref 1 00000100:00100000:2.0:1789760791.371248:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721481818820 time 721495854832 state 0x0 ref 1 00000100:00100000:2.0:1789760791.371259:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc11c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503899136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.371316:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80306176, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264c8 00010000:00010000:2.0:1789760791.371326:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.371342:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.371346:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.371352:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.371365:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.378158:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986913 is committed 00002000:00010000:2.0:1789760791.378910:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80306176 -> 80310272 00002000:00010000:2.0:1789760791.378916:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156872 -> 156880 00010000:00010000:2.0:1789760791.378929:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378942:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378951:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378963:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.378968:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378977:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378995:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.379002:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2023 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80306176->80310271] (req 80306176->80310271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.379138:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc11c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503899136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7874us (8116us total) trans 4294986913 rc 0/0 00000100:00100000:2.0:1789760791.379157:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19617 00000100:00100000:2.0:1789760791.379167:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721481818820 time 721495854832 state 0x2 ref 0 00000100:00100000:3.0:1789760791.384958:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.385052:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899264 00000100:00100000:0.0:1789760791.385064:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899264 00000100:00100000:0.0:1789760791.385127:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721481818820 time 721495854832 state 0x0 ref 1 00000100:00100000:0.0:1789760791.385142:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721496115248 time 721495854832 state 0x1 ref 1 00000100:00100000:0.0:1789760791.391554:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721496115248 time 721516152338 state 0x0 ref 1 00000100:00100000:0.0:1789760791.391584:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503899264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.391642:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80310272, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264d0 00010000:00010000:0.0:1789760791.391651:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.391667:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.391671:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.391678:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.391692:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.402696:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986914 is committed 00002000:00010000:1.0:1789760791.402812:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80310272 -> 80314368 00002000:00010000:1.0:1789760791.402817:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156880 -> 156888 00010000:00010000:1.0:1789760791.402830:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.402843:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.402853:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.402860:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.402865:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.402874:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.402894:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.402902:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f202a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80310272->80314367] (req 80310272->80314367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.402977:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503899264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11404us (18024us total) trans 4294986914 rc 0/0 00000100:00100000:1.0:1789760791.402986:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19618 00000100:00100000:1.0:1789760791.402996:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721496115248 time 721516152338 state 0x2 ref 0 00000100:00100000:0.0:1789760791.409886:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.409979:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899392 00000100:00100000:1.0:1789760791.409990:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899392 00000100:00100000:1.0:1789760791.410045:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721496115248 time 721516152338 state 0x0 ref 1 00000100:00100000:1.0:1789760791.410059:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721516412754 time 721516152338 state 0x1 ref 1 00000100:00100000:1.0:1789760791.410089:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721516412754 time 721534695786 state 0x0 ref 1 00000100:00100000:1.0:1789760791.410102:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503899392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.410155:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80314368, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264d8 00010000:00010000:1.0:1789760791.410164:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.410179:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.410185:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.410191:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.410204:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.416946:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986915 is committed 00002000:00010000:1.0:1789760791.417057:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80314368 -> 80318464 00002000:00010000:1.0:1789760791.417063:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156888 -> 156896 00010000:00010000:1.0:1789760791.417075:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.417089:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.417099:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.417106:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.417111:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.417119:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.417138:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.417151:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2031 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80314368->80318463] (req 80314368->80318463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.417328:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503899392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7223us (7446us total) trans 4294986915 rc 0/0 00000100:00100000:1.0:1789760791.417339:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19619 00000100:00100000:1.0:1789760791.417349:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721516412754 time 721534695786 state 0x2 ref 0 00000100:00100000:2.0:1789760791.420943:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.421003:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899520 00000100:00100000:1.0:1789760791.421012:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899520 00000100:00100000:1.0:1789760791.421063:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721516412754 time 721534695786 state 0x0 ref 1 00000100:00100000:1.0:1789760791.421078:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721534956202 time 721534695786 state 0x1 ref 1 00000100:00100000:1.0:1789760791.421107:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721534956202 time 721545713534 state 0x0 ref 1 00000100:00100000:1.0:1789760791.421120:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503899520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.421168:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80318464, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264e0 00010000:00010000:1.0:1789760791.421177:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.421194:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.421198:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.421204:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.421223:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.426342:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986916 is committed 00002000:00010000:2.0:1789760791.426445:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80318464 -> 80322560 00002000:00010000:2.0:1789760791.426451:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156896 -> 156904 00010000:00010000:2.0:1789760791.426464:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.426483:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.426491:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.426509:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.426514:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.426522:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.426550:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.426564:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2038 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80318464->80322559] (req 80318464->80322559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.426656:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503899520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5532us (5716us total) trans 4294986916 rc 0/0 00000100:00100000:2.0:1789760791.426666:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19620 00000100:00100000:2.0:1789760791.426675:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721534956202 time 721545713534 state 0x2 ref 0 00000100:00100000:2.0:1789760791.430299:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.430433:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899648 00000100:00100000:1.0:1789760791.430444:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899648 00000100:00100000:1.0:1789760791.430508:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721534956202 time 721545713534 state 0x0 ref 1 00000100:00100000:1.0:1789760791.430522:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721545973950 time 721545713534 state 0x1 ref 1 00000100:00100000:1.0:1789760791.430561:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721545973950 time 721555161117 state 0x0 ref 1 00000100:00100000:1.0:1789760791.430574:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503899648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.430622:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80322560, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264e8 00010000:00010000:1.0:1789760791.430632:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.430653:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.430657:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.430664:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.430678:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.435851:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986917 is committed 00002000:00010000:1.0:1789760791.435970:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80322560 -> 80326656 00002000:00010000:1.0:1789760791.435976:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156904 -> 156912 00010000:00010000:1.0:1789760791.435989:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.436002:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.436012:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.436020:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.436024:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.436034:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.436052:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.436060:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f203f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80322560->80326655] (req 80322560->80326655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.436190:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503899648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5612us (5895us total) trans 4294986917 rc 0/0 00000100:00100000:1.0:1789760791.436202:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19621 00000100:00100000:1.0:1789760791.436213:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721545973950 time 721555161117 state 0x2 ref 0 00000100:00100000:1.0:1789760791.440207:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.440307:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899776 00000100:00100000:3.0:1789760791.440316:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899776 00000100:00100000:3.0:1789760791.443800:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721545973950 time 721555161117 state 0x0 ref 1 00000100:00100000:3.0:1789760791.443829:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721555421533 time 721555161117 state 0x1 ref 1 00000100:00100000:3.0:1789760791.443907:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721555421533 time 721568512670 state 0x0 ref 1 00000100:00100000:3.0:1789760791.443929:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503899776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.443991:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80326656, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264f0 00010000:00010000:3.0:1789760791.444001:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.444018:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.444024:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.444031:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.444046:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.454098:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986918 is committed 00002000:00010000:3.0:1789760791.454240:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80326656 -> 80330752 00002000:00010000:3.0:1789760791.454245:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156912 -> 156920 00010000:00010000:3.0:1789760791.454258:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.454272:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.454280:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.454288:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.454292:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.454300:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.454321:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.454336:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2046 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80326656->80330751] (req 80326656->80330751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.454480:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503899776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10557us (14278us total) trans 4294986918 rc 0/0 00000100:00100000:3.0:1789760791.454489:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19622 00000100:00100000:3.0:1789760791.454509:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721555421533 time 721568512670 state 0x2 ref 0 00000100:00100000:2.0:1789760791.463040:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.463156:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503899904 00000100:00100000:1.0:1789760791.463166:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503899904 00000100:00100000:1.0:1789760791.463220:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721555421533 time 721568512670 state 0x0 ref 1 00000100:00100000:1.0:1789760791.463235:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721568773086 time 721568512670 state 0x1 ref 1 00000100:00100000:1.0:1789760791.463275:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721568773086 time 721587881521 state 0x0 ref 1 00000100:00100000:1.0:1789760791.463288:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503899904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.463335:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80330752, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x264f8 00010000:00010000:1.0:1789760791.463344:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.463359:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.463363:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.463369:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.463383:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.470864:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986919 is committed 00002000:00010000:1.0:1789760791.470991:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80330752 -> 80334848 00002000:00010000:1.0:1789760791.470996:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156920 -> 156928 00010000:00010000:1.0:1789760791.471009:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.471022:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.471031:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.471039:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.471043:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.471053:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.471073:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.471081:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f204d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80330752->80334847] (req 80330752->80334847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.471181:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503899904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7891us (8146us total) trans 4294986919 rc 0/0 00000100:00100000:1.0:1789760791.471192:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19623 00000100:00100000:1.0:1789760791.471202:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721568773086 time 721587881521 state 0x2 ref 0 00000100:00100000:2.0:1789760791.477724:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.478841:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900032 00000100:00100000:1.0:1789760791.478856:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900032 00000100:00100000:1.0:1789760791.479673:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721568773086 time 721587881521 state 0x0 ref 1 00000100:00100000:1.0:1789760791.479689:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721588141937 time 721587881521 state 0x1 ref 1 00000100:00100000:1.0:1789760791.479726:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721588141937 time 721604333161 state 0x0 ref 1 00000100:00100000:1.0:1789760791.479741:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.479791:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80334848, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26500 00010000:00010000:1.0:1789760791.479801:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.479817:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.479821:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.479827:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.479841:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.486799:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986920 is committed 00002000:00010000:1.0:1789760791.487457:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80334848 -> 80338944 00002000:00010000:1.0:1789760791.487464:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156928 -> 156936 00010000:00010000:1.0:1789760791.487478:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.487511:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.487520:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.487531:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.487536:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.487554:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.487574:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.487581:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2054 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80334848->80338943] (req 80334848->80338943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.487680:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7939us (9959us total) trans 4294986920 rc 0/0 00000100:00100000:1.0:1789760791.487696:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19624 00000100:00100000:1.0:1789760791.487706:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721588141937 time 721604333161 state 0x2 ref 0 00000100:00100000:0.0:1789760791.492037:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.492182:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900160 00000100:00100000:2.0:1789760791.492192:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900160 00000100:00100000:2.0:1789760791.492248:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721588141937 time 721604333161 state 0x0 ref 1 00000100:00100000:2.0:1789760791.492263:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721604593577 time 721604333161 state 0x1 ref 1 00000100:00100000:2.0:1789760791.492294:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721604593577 time 721616900311 state 0x0 ref 1 00000100:00100000:2.0:1789760791.492306:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.492354:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80338944, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26508 00010000:00010000:2.0:1789760791.492363:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.492378:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.492382:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.492425:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.492441:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.498815:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986921 is committed 00002000:00010000:2.0:1789760791.500308:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80338944 -> 80343040 00002000:00010000:2.0:1789760791.500317:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156936 -> 156944 00010000:00010000:2.0:1789760791.500331:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.500344:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.500354:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.500362:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.500367:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.500375:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.500407:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.500416:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f205b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80338944->80343039] (req 80338944->80343039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.500530:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8220us (8497us total) trans 4294986921 rc 0/0 00000100:00100000:2.0:1789760791.500550:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19625 00000100:00100000:2.0:1789760791.500560:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721604593577 time 721616900311 state 0x2 ref 0 00000100:00100000:2.0:1789760791.514820:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.515289:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900288 00000100:00100000:3.0:1789760791.515301:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900288 00000100:00100000:3.0:1789760791.515356:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721604593577 time 721616900311 state 0x0 ref 1 00000100:00100000:3.0:1789760791.515372:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721617160727 time 721616900311 state 0x1 ref 1 00000100:00100000:3.0:1789760791.515492:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721617160727 time 721640089795 state 0x0 ref 1 00000100:00100000:3.0:1789760791.515520:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.515755:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80343040, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26510 00010000:00010000:3.0:1789760791.515765:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.515795:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.515800:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.515805:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.515825:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.525036:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986922 is committed 00002000:00010000:3.0:1789760791.525209:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80343040 -> 80347136 00002000:00010000:3.0:1789760791.525214:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156944 -> 156952 00010000:00010000:3.0:1789760791.525226:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.525239:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.525248:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.525255:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.525260:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.525268:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.525287:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.525295:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2062 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80343040->80347135] (req 80343040->80347135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.525387:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9866us (10572us total) trans 4294986922 rc 0/0 00000100:00100000:3.0:1789760791.525446:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19626 00000100:00100000:3.0:1789760791.525456:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721617160727 time 721640089795 state 0x2 ref 0 00000100:00100000:0.0:1789760791.529858:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.530861:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900416 00000100:00100000:1.0:1789760791.530873:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900416 00000100:00100000:1.0:1789760791.530942:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721617160727 time 721640089795 state 0x0 ref 1 00000100:00100000:1.0:1789760791.530968:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721640350211 time 721640089795 state 0x1 ref 1 00000100:00100000:1.0:1789760791.530998:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721640350211 time 721655603986 state 0x0 ref 1 00000100:00100000:1.0:1789760791.531011:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.531058:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80347136, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26518 00010000:00010000:1.0:1789760791.531066:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.531083:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.531087:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.531093:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.531107:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.540920:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986923 is committed 00002000:00010000:2.0:1789760791.541037:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80347136 -> 80351232 00002000:00010000:2.0:1789760791.541042:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156952 -> 156960 00010000:00010000:2.0:1789760791.541056:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.541070:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.541079:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.541086:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.541091:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.541099:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.541118:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.541126:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2069 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80347136->80351231] (req 80347136->80351231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.541341:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10326us (11486us total) trans 4294986923 rc 0/0 00000100:00100000:2.0:1789760791.541353:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19627 00000100:00100000:2.0:1789760791.541372:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721640350211 time 721655603986 state 0x2 ref 0 00000100:00100000:0.0:1789760791.551742:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.552053:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900544 00000100:00100000:1.0:1789760791.552063:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900544 00000100:00100000:1.0:1789760791.552117:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721640350211 time 721655603986 state 0x0 ref 1 00000100:00100000:1.0:1789760791.552137:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721655864402 time 721655603986 state 0x1 ref 1 00000100:00100000:1.0:1789760791.552168:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721655864402 time 721676774817 state 0x0 ref 1 00000100:00100000:1.0:1789760791.552180:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.552237:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80351232, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26520 00010000:00010000:1.0:1789760791.552247:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.552263:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.552267:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.552283:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.552298:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.561264:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986924 is committed 00002000:00010000:1.0:1789760791.561726:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80351232 -> 80355328 00002000:00010000:1.0:1789760791.561732:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156960 -> 156968 00010000:00010000:1.0:1789760791.561746:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.561764:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.561772:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.561779:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.561784:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.561861:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.561882:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.561890:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2070 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80351232->80355327] (req 80351232->80355327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.562021:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9838us (10284us total) trans 4294986924 rc 0/0 00000100:00100000:1.0:1789760791.562031:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19628 00000100:00100000:1.0:1789760791.562042:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721655864402 time 721676774817 state 0x2 ref 0 00000100:00100000:0.0:1789760791.567092:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.567539:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900672 00000100:00100000:2.0:1789760791.567550:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900672 00000100:00100000:2.0:1789760791.567613:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721655864402 time 721676774817 state 0x0 ref 1 00000100:00100000:2.0:1789760791.567629:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721677035233 time 721676774817 state 0x1 ref 1 00000100:00100000:2.0:1789760791.567658:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721677035233 time 721692264992 state 0x0 ref 1 00000100:00100000:2.0:1789760791.567671:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.567717:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80355328, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26528 00010000:00010000:2.0:1789760791.567726:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.567742:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.567747:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.567753:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.567779:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.574125:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986925 is committed 00002000:00010000:2.0:1789760791.574889:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80355328 -> 80359424 00002000:00010000:2.0:1789760791.574896:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156968 -> 156976 00010000:00010000:2.0:1789760791.574911:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.574923:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.574932:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.574939:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.574944:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.574953:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.574973:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.574981:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2077 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80355328->80359423] (req 80355328->80359423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.575119:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7445us (8030us total) trans 4294986925 rc 0/0 00000100:00100000:2.0:1789760791.575129:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19629 00000100:00100000:2.0:1789760791.575138:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721677035233 time 721692264992 state 0x2 ref 0 00000100:00100000:0.0:1789760791.585927:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.587742:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900800 00000100:00100000:3.0:1789760791.587757:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900800 00000100:00100000:3.0:1789760791.587942:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721677035233 time 721692264992 state 0x0 ref 1 00000100:00100000:3.0:1789760791.588026:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721692525408 time 721692264992 state 0x1 ref 1 00000100:00100000:3.0:1789760791.588062:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721692525408 time 721712669421 state 0x0 ref 1 00000100:00100000:3.0:1789760791.588076:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.588136:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80359424, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26530 00010000:00010000:3.0:1789760791.588147:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.588166:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.588172:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.588178:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.588194:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.597005:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986926 is committed 00002000:00010000:3.0:1789760791.597185:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80359424 -> 80363520 00002000:00010000:3.0:1789760791.597191:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156976 -> 156984 00010000:00010000:3.0:1789760791.597202:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.597226:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.597234:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.597241:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.597245:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.597254:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.597272:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.597281:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f207e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80359424->80363519] (req 80359424->80363519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.597371:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503900800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9293us (11450us total) trans 4294986926 rc 0/0 00000100:00100000:3.0:1789760791.597382:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19630 00000100:00100000:3.0:1789760791.597445:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721692525408 time 721712669421 state 0x2 ref 0 00000100:00100000:0.0:1789760791.606231:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.606330:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503900928 00000100:00100000:1.0:1789760791.606342:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503900928 00000100:00100000:1.0:1789760791.606437:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721692525408 time 721712669421 state 0x0 ref 1 00000100:00100000:1.0:1789760791.606452:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721712929837 time 721712669421 state 0x1 ref 1 00000100:00100000:1.0:1789760791.606487:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721712929837 time 721731094398 state 0x0 ref 1 00000100:00100000:1.0:1789760791.606508:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.606554:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80363520, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26538 00010000:00010000:1.0:1789760791.606562:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606578:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.606582:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.606589:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.606602:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.612653:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986927 is committed 00002000:00010000:2.0:1789760791.612760:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80363520 -> 80367616 00002000:00010000:2.0:1789760791.612765:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156984 -> 156992 00010000:00010000:2.0:1789760791.612779:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.612846:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.612856:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.612863:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.612867:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.612877:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.612898:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.612906:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2085 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80363520->80367615] (req 80363520->80367615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.613051:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503900928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6540us (6822us total) trans 4294986927 rc 0/0 00000100:00100000:2.0:1789760791.613066:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19631 00000100:00100000:2.0:1789760791.613076:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721712929837 time 721731094398 state 0x2 ref 0 00000100:00100000:1.0:1789760791.619963:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.620297:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901056 00000100:00100000:2.0:1789760791.620308:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901056 00000100:00100000:2.0:1789760791.620362:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721712929837 time 721731094398 state 0x0 ref 1 00000100:00100000:2.0:1789760791.620376:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721731354814 time 721731094398 state 0x1 ref 1 00000100:00100000:2.0:1789760791.620471:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721731354814 time 721745076979 state 0x0 ref 1 00000100:00100000:2.0:1789760791.620484:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503901056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.620749:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80367616, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26540 00010000:00010000:2.0:1789760791.620759:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.620775:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.620779:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.620999:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.621017:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.630093:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986928 is committed 00002000:00010000:2.0:1789760791.630343:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80367616 -> 80371712 00002000:00010000:2.0:1789760791.630348:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 156992 -> 157000 00010000:00010000:2.0:1789760791.630360:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.630372:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.630380:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.630386:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.630427:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.630436:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.630453:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.630459:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f208c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80367616->80371711] (req 80367616->80371711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.630560:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503901056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10073us (10603us total) trans 4294986928 rc 0/0 00000100:00100000:2.0:1789760791.630569:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19632 00000100:00100000:2.0:1789760791.630579:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721731354814 time 721745076979 state 0x2 ref 0 00000100:00100000:0.0:1789760791.641839:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.642685:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901184 00000100:00100000:2.0:1789760791.642698:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901184 00000100:00100000:2.0:1789760791.642753:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721731354814 time 721745076979 state 0x0 ref 1 00000100:00100000:2.0:1789760791.642769:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721745337395 time 721745076979 state 0x1 ref 1 00000100:00100000:2.0:1789760791.642805:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721745337395 time 721767411853 state 0x0 ref 1 00000100:00100000:2.0:1789760791.642819:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503901184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.642867:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80371712, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26548 00010000:00010000:2.0:1789760791.642877:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.642892:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.642895:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.642901:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.642915:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.649484:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986929 is committed 00002000:00010000:1.0:1789760791.649709:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80371712 -> 80375808 00002000:00010000:1.0:1789760791.649714:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157000 -> 157008 00010000:00010000:1.0:1789760791.649728:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.649740:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.649754:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.649761:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.649766:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.649774:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.649801:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.649810:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2093 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80371712->80375807] (req 80371712->80375807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.649898:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503901184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7077us (8064us total) trans 4294986929 rc 0/0 00000100:00100000:1.0:1789760791.649908:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19633 00000100:00100000:1.0:1789760791.649918:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721745337395 time 721767411853 state 0x2 ref 0 00000100:00100000:0.0:1789760791.658484:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.658633:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901312 00000100:00100000:3.0:1789760791.658643:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901312 00000100:00100000:3.0:1789760791.658697:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721745337395 time 721767411853 state 0x0 ref 1 00000100:00100000:3.0:1789760791.658716:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721767672269 time 721767411853 state 0x1 ref 1 00000100:00100000:3.0:1789760791.658748:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721767672269 time 721783355289 state 0x0 ref 1 00000100:00100000:3.0:1789760791.658760:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503901312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.658806:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80375808, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26550 00010000:00010000:3.0:1789760791.658816:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.658831:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.658835:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.658841:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.658854:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.665725:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986930 is committed 00002000:00010000:3.0:1789760791.665924:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80375808 -> 80379904 00002000:00010000:3.0:1789760791.665929:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157008 -> 157016 00010000:00010000:3.0:1789760791.665942:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.665953:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.665961:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.665968:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760791.665973:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.665981:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.666109:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.666118:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f209a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80375808->80379903] (req 80375808->80379903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.666252:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503901312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7488us (7771us total) trans 4294986930 rc 0/0 00000100:00100000:3.0:1789760791.666263:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19634 00000100:00100000:3.0:1789760791.666272:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721767672269 time 721783355289 state 0x2 ref 0 00000100:00100000:1.0:1789760791.671120:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.671202:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901440 00000100:00100000:3.0:1789760791.671211:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901440 00000100:00100000:3.0:1789760791.671378:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721767672269 time 721783355289 state 0x0 ref 1 00000100:00100000:3.0:1789760791.671420:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721783615705 time 721783355289 state 0x1 ref 1 00000100:00100000:3.0:1789760791.671490:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721783615705 time 721796096518 state 0x0 ref 1 00000100:00100000:3.0:1789760791.671515:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503901440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.671573:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80379904, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26558 00010000:00010000:3.0:1789760791.671583:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.671599:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.671604:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.671611:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.671625:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.677541:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986931 is committed 00002000:00010000:0.0:1789760791.677752:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80379904 -> 80384000 00002000:00010000:0.0:1789760791.677758:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157016 -> 157024 00010000:00010000:0.0:1789760791.677774:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.677788:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.677798:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.677805:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.677810:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.677819:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.677839:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.677847:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f20a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80379904->80383999] (req 80379904->80383999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.677947:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503901440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6431us (6833us total) trans 4294986931 rc 0/0 00000100:00100000:0.0:1789760791.677958:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19635 00000100:00100000:0.0:1789760791.677968:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721783615705 time 721796096518 state 0x2 ref 0 00000100:00100000:0.0:1789760791.681549:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.681627:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901568 00000100:00100000:2.0:1789760791.681636:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901568 00000100:00100000:2.0:1789760791.681691:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721783615705 time 721796096518 state 0x0 ref 1 00000100:00100000:2.0:1789760791.681706:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721796356934 time 721796096518 state 0x1 ref 1 00000100:00100000:2.0:1789760791.681727:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721796356934 time 721806334911 state 0x0 ref 1 00000100:00100000:2.0:1789760791.681738:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503901568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.681784:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80384000, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26560 00010000:00010000:2.0:1789760791.681793:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681808:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.681812:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.681818:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681832:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.686144:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986932 is committed 00002000:00010000:2.0:1789760791.686341:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80384000 -> 80388096 00002000:00010000:2.0:1789760791.686346:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157024 -> 157032 00010000:00010000:2.0:1789760791.686358:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.686370:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.686379:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.686451:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.686456:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.686465:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.686485:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.686503:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80384000->80388095] (req 80384000->80388095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.686744:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503901568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5001us (5348us total) trans 4294986932 rc 0/0 00000100:00100000:2.0:1789760791.686754:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19636 00000100:00100000:2.0:1789760791.686765:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721796356934 time 721806334911 state 0x2 ref 0 00000100:00100000:1.0:1789760791.690843:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.690925:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901696 00000100:00100000:2.0:1789760791.690936:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901696 00000100:00100000:2.0:1789760791.690990:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721796356934 time 721806334911 state 0x0 ref 1 00000100:00100000:2.0:1789760791.691015:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721806595327 time 721806334911 state 0x1 ref 1 00000100:00100000:2.0:1789760791.691044:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721806595327 time 721815650636 state 0x0 ref 1 00000100:00100000:2.0:1789760791.691059:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503901696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.691103:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80388096, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26568 00010000:00010000:2.0:1789760791.691111:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.691126:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.691131:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.691137:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.691151:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.696865:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986933 is committed 00002000:00010000:2.0:1789760791.696956:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80388096 -> 80392192 00002000:00010000:2.0:1789760791.696962:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157032 -> 157040 00010000:00010000:2.0:1789760791.696973:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.696986:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.696995:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.697011:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.697017:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.697026:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.697044:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.697051:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f20af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80388096->80392191] (req 80388096->80392191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.697136:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503901696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6074us (6298us total) trans 4294986933 rc 0/0 00000100:00100000:2.0:1789760791.697146:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19637 00000100:00100000:2.0:1789760791.697156:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721806595327 time 721815650636 state 0x2 ref 0 00000100:00100000:1.0:1789760791.700657:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.700774:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901824 00000100:00100000:3.0:1789760791.700784:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901824 00000100:00100000:3.0:1789760791.700842:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721806595327 time 721815650636 state 0x0 ref 1 00000100:00100000:3.0:1789760791.700855:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721815911052 time 721815650636 state 0x1 ref 1 00000100:00100000:3.0:1789760791.700887:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721815911052 time 721825494017 state 0x0 ref 1 00000100:00100000:3.0:1789760791.700900:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503901824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.700964:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80392192, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26570 00010000:00010000:3.0:1789760791.700974:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.700990:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.700995:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.701001:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.701015:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.714445:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986934 is committed 00002000:00010000:0.0:1789760791.716165:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80392192 -> 80396288 00002000:00010000:0.0:1789760791.716174:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157040 -> 157048 00010000:00010000:0.0:1789760791.716186:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.716199:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.716209:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.716280:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.716286:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.716297:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.716317:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.716324:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f20b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80392192->80396287] (req 80392192->80396287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.716484:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503901824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15581us (15944us total) trans 4294986934 rc 0/0 00000100:00100000:0.0:1789760791.716503:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19638 00000100:00100000:0.0:1789760791.716512:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721815911052 time 721825494017 state 0x2 ref 0 00000100:00100000:0.0:1789760791.721642:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760791.721788:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503901952 00000100:00100000:3.0:1789760791.721799:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503901952 00000100:00100000:3.0:1789760791.721856:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721815911052 time 721825494017 state 0x0 ref 1 00000100:00100000:3.0:1789760791.721870:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721825754433 time 721825494017 state 0x1 ref 1 00000100:00100000:3.0:1789760791.721895:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721825754433 time 721846501441 state 0x0 ref 1 00000100:00100000:3.0:1789760791.721906:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503901952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760791.721954:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80396288, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26578 00010000:00010000:3.0:1789760791.721963:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.721980:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760791.721984:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.721990:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.722012:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.730271:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986935 is committed 00002000:00010000:1.0:1789760791.730431:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80396288 -> 80400384 00002000:00010000:1.0:1789760791.730437:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157048 -> 157056 00010000:00010000:1.0:1789760791.730452:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730465:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730474:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730482:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.730487:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730507:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730528:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730546:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f20bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80396288->80400383] (req 80396288->80400383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.730662:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503901952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8751us (9025us total) trans 4294986935 rc 0/0 00000100:00100000:1.0:1789760791.730673:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19639 00000100:00100000:1.0:1789760791.730683:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721825754433 time 721846501441 state 0x2 ref 0 00000100:00100000:3.0:1789760791.736215:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.736369:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902080 00000100:00100000:2.0:1789760791.736379:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902080 00000100:00100000:2.0:1789760791.736469:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721825754433 time 721846501441 state 0x0 ref 1 00000100:00100000:2.0:1789760791.736484:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721846761857 time 721846501441 state 0x1 ref 1 00000100:00100000:2.0:1789760791.736516:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721846761857 time 721861122310 state 0x0 ref 1 00000100:00100000:2.0:1789760791.736528:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503902080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.736584:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80400384, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26580 00010000:00010000:2.0:1789760791.736593:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.736608:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.736612:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.736618:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.736630:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.741457:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986936 is committed 00002000:00010000:2.0:1789760791.741670:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80400384 -> 80404480 00002000:00010000:2.0:1789760791.741676:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157056 -> 157064 00010000:00010000:2.0:1789760791.741690:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.741703:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.741712:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.741719:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.741723:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.741731:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.741750:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.741757:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80400384->80404479] (req 80400384->80404479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.741847:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503902080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5316us (5636us total) trans 4294986936 rc 0/0 00000100:00100000:2.0:1789760791.741858:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19640 00000100:00100000:2.0:1789760791.741867:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721846761857 time 721861122310 state 0x2 ref 0 00000100:00100000:2.0:1789760791.747863:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.747986:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902208 00000100:00100000:1.0:1789760791.747997:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902208 00000100:00100000:1.0:1789760791.748062:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721846761857 time 721861122310 state 0x0 ref 1 00000100:00100000:1.0:1789760791.748078:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721861382726 time 721861122310 state 0x1 ref 1 00000100:00100000:1.0:1789760791.748113:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721861382726 time 721872720282 state 0x0 ref 1 00000100:00100000:1.0:1789760791.748125:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503902208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.748175:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80404480, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26588 00010000:00010000:1.0:1789760791.748185:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.748201:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.748205:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.748212:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.748226:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.754801:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986937 is committed 00002000:00010000:0.0:1789760791.759648:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80404480 -> 80408576 00002000:00010000:0.0:1789760791.759659:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157064 -> 157072 00010000:00010000:0.0:1789760791.759675:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.759688:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.759698:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.759706:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.759710:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.759719:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.759741:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.759748:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f20cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80404480->80408575] (req 80404480->80408575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.759857:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503902208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11728us (11999us total) trans 4294986937 rc 0/0 00000100:00100000:0.0:1789760791.759867:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19641 00000100:00100000:0.0:1789760791.759875:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721861382726 time 721872720282 state 0x2 ref 0 00000100:00100000:3.0:1789760791.762900:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6968000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523614976:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789760791.762988:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760791.763120:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523614976 00000100:00100000:2.0:1789760791.763130:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523614976 00000100:00100000:2.0:1789760791.763183:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1789760791.763193:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523614976:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760791.763228:0:6518:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:0.0:1789760791.769799:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.770689:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902336 00000100:00100000:1.0:1789760791.770701:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902336 00000100:00100000:1.0:1789760791.770759:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721861382726 time 721872720282 state 0x0 ref 1 00000100:00100000:1.0:1789760791.770775:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721872980698 time 721872720282 state 0x1 ref 1 00000100:00100000:1.0:1789760791.770814:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721872980698 time 721895416139 state 0x0 ref 1 00000100:00100000:1.0:1789760791.770826:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503902336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.770877:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80408576, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26590 00010000:00010000:1.0:1789760791.770887:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.770903:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.770907:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.770914:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.771248:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.776781:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986938 is committed 00002000:00010000:1.0:1789760791.777416:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80408576 -> 80412672 00002000:00010000:1.0:1789760791.777431:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157072 -> 157080 00010000:00010000:1.0:1789760791.777445:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.777464:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.777472:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.777479:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.777484:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.777504:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.777535:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.777543:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80408576->80412671] (req 80408576->80412671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.779161:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523614976:12345-0@lo:41:osp-pre-1-0.0 Request processed in 15961us (16164us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760791.779175:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:3.0:1789760791.779647:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6968000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523614976:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760791.779921:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503902336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9092us (10126us total) trans 4294986938 rc 0/0 00000100:00100000:1.0:1789760791.779933:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19642 00000100:00100000:1.0:1789760791.779943:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721872980698 time 721895416139 state 0x2 ref 0 00000100:00100000:3.0:1789760791.782730:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.782896:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902464 00000100:00100000:2.0:1789760791.782906:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902464 00000100:00100000:2.0:1789760791.782960:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721872980698 time 721895416139 state 0x0 ref 1 00000100:00100000:2.0:1789760791.782974:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721895676555 time 721895416139 state 0x1 ref 1 00000100:00100000:2.0:1789760791.783011:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721895676555 time 721907617887 state 0x0 ref 1 00000100:00100000:2.0:1789760791.783023:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503902464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.783069:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80412672, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x26598 00010000:00010000:2.0:1789760791.783079:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.783094:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760791.783098:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.783105:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.783120:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.789256:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986939 is committed 00002000:00010000:2.0:1789760791.789383:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80412672 -> 80416768 00002000:00010000:2.0:1789760791.789413:0:15091:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:2.0:1789760791.789623:0:15091:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00002000:00010000:2.0:1789760791.789627:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157080 -> 157088 00010000:00010000:2.0:1789760791.789642:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.789654:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.789663:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.789670:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.789674:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.789684:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.789704:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.789712:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f20d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80412672->80416767] (req 80412672->80416767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.789933:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503902464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6906us (7206us total) trans 4294986939 rc 0/0 00000100:00100000:0.0:1789760791.789944:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19643 00000100:00100000:0.0:1789760791.789954:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721895676555 time 721907617887 state 0x2 ref 0 00000100:00100000:3.0:1789760791.795801:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.795894:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902592 00000100:00100000:0.0:1789760791.795903:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902592 00000100:00100000:0.0:1789760791.795955:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721895676555 time 721907617887 state 0x0 ref 1 00000100:00100000:0.0:1789760791.795968:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721907878303 time 721907617887 state 0x1 ref 1 00000100:00100000:0.0:1789760791.796008:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721907878303 time 721920607546 state 0x0 ref 1 00000100:00100000:0.0:1789760791.796020:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503902592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.796066:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80416768, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265a0 00010000:00010000:0.0:1789760791.796074:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.796094:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.796099:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.796105:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.796118:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.801313:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986940 is committed 00002000:00010000:0.0:1789760791.801417:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80416768 -> 80420864 00002000:00010000:0.0:1789760791.801423:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157088 -> 157096 00010000:00010000:0.0:1789760791.801436:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.801449:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.801458:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.801466:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.801471:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.801480:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.801516:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.801537:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f20e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80416768->80420863] (req 80416768->80420863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.801630:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503902592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5607us (5832us total) trans 4294986940 rc 0/0 00000100:00100000:0.0:1789760791.801640:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19644 00000100:00100000:0.0:1789760791.801649:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721907878303 time 721920607546 state 0x2 ref 0 00000100:00100000:2.0:1789760791.807936:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.808132:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902720 00000100:00100000:0.0:1789760791.808141:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902720 00000100:00100000:0.0:1789760791.808189:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721907878303 time 721920607546 state 0x0 ref 1 00000100:00100000:0.0:1789760791.808204:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721920867962 time 721920607546 state 0x1 ref 1 00000100:00100000:0.0:1789760791.808224:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721920867962 time 721932831650 state 0x0 ref 1 00000100:00100000:0.0:1789760791.808237:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503902720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.808282:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80420864, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265a8 00010000:00010000:0.0:1789760791.808291:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.808307:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.808312:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.808318:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.808332:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.817595:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986941 is committed 00002000:00010000:0.0:1789760791.817960:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80420864 -> 80424960 00002000:00010000:0.0:1789760791.817966:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157096 -> 157104 00010000:00010000:0.0:1789760791.817979:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.817992:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.818001:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.818008:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.818013:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.818023:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.818043:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.818056:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80420864->80424959] (req 80420864->80424959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.818153:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503902720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9913us (10222us total) trans 4294986941 rc 0/0 00000100:00100000:0.0:1789760791.818163:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19645 00000100:00100000:0.0:1789760791.818179:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721920867962 time 721932831650 state 0x2 ref 0 00000100:00100000:3.0:1789760791.824135:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.825814:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902848 00000100:00100000:1.0:1789760791.825829:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902848 00000100:00100000:1.0:1789760791.826106:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721920867962 time 721932831650 state 0x0 ref 1 00000100:00100000:1.0:1789760791.826124:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721933092066 time 721932831650 state 0x1 ref 1 00000100:00100000:1.0:1789760791.826161:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721933092066 time 721950767735 state 0x0 ref 1 00000100:00100000:1.0:1789760791.826175:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503902848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.826223:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80424960, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265b0 00010000:00010000:1.0:1789760791.826233:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.826249:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.826254:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.826261:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.826275:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.835462:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986942 is committed 00002000:00010000:1.0:1789760791.836479:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80424960 -> 80429056 00002000:00010000:1.0:1789760791.836489:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157104 -> 157112 00010000:00010000:1.0:1789760791.836515:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.836536:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.836551:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.836559:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.836564:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.836572:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.836591:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.836598:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f20ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80424960->80429055] (req 80424960->80429055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.836699:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503902848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10521us (12567us total) trans 4294986942 rc 0/0 00000100:00100000:1.0:1789760791.836710:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19646 00000100:00100000:1.0:1789760791.836719:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721933092066 time 721950767735 state 0x2 ref 0 00000100:00100000:3.0:1789760791.841227:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.841302:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503902976 00000100:00100000:1.0:1789760791.841317:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503902976 00000100:00100000:1.0:1789760791.841378:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721933092066 time 721950767735 state 0x0 ref 1 00000100:00100000:1.0:1789760791.841443:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721951028151 time 721950767735 state 0x1 ref 1 00000100:00100000:1.0:1789760791.841468:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721951028151 time 721966074858 state 0x0 ref 1 00000100:00100000:1.0:1789760791.841481:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503902976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.841544:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80429056, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265b8 00010000:00010000:1.0:1789760791.841554:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.841569:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760791.841574:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.841580:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.841594:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.850819:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986943 is committed 00002000:00010000:1.0:1789760791.851009:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80429056 -> 80433152 00002000:00010000:1.0:1789760791.851015:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157112 -> 157120 00010000:00010000:1.0:1789760791.851028:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851040:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851048:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851062:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.851066:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851074:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851092:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851100:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f20f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80429056->80433151] (req 80429056->80433151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.851190:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503902976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9707us (9972us total) trans 4294986943 rc 0/0 00000100:00100000:1.0:1789760791.851200:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19647 00000100:00100000:1.0:1789760791.851209:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721951028151 time 721966074858 state 0x2 ref 0 00000100:00100000:2.0:1789760791.854902:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.855519:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503903104 00000100:00100000:0.0:1789760791.855545:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503903104 00000100:00100000:0.0:1789760791.855608:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721951028151 time 721966074858 state 0x0 ref 1 00000100:00100000:0.0:1789760791.855625:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721966335274 time 721966074858 state 0x1 ref 1 00000100:00100000:0.0:1789760791.855658:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721966335274 time 721980264387 state 0x0 ref 1 00000100:00100000:0.0:1789760791.855672:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.855716:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80433152, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265c0 00010000:00010000:0.0:1789760791.855726:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.855747:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.855751:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.855758:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.855778:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.863298:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986944 is committed 00002000:00010000:0.0:1789760791.863699:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80433152 -> 80437248 00002000:00010000:0.0:1789760791.863705:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157120 -> 157128 00010000:00010000:0.0:1789760791.863719:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.863732:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.863741:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.863748:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.863753:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.863763:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.863854:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.863864:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f20fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80433152->80437247] (req 80433152->80437247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.864002:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8328us (9109us total) trans 4294986944 rc 0/0 00000100:00100000:0.0:1789760791.864013:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19648 00000100:00100000:0.0:1789760791.864023:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721966335274 time 721980264387 state 0x2 ref 0 00000100:00100000:3.0:1789760791.868362:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.868462:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503903232 00000100:00100000:0.0:1789760791.868472:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503903232 00000100:00100000:0.0:1789760791.868549:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721966335274 time 721980264387 state 0x0 ref 1 00000100:00100000:0.0:1789760791.868573:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721980524803 time 721980264387 state 0x1 ref 1 00000100:00100000:0.0:1789760791.868601:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721980524803 time 721993207398 state 0x0 ref 1 00000100:00100000:0.0:1789760791.868612:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.868660:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80437248, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265c8 00010000:00010000:0.0:1789760791.868669:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.868685:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760791.868690:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.868696:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.868709:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.874678:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986945 is committed 00002000:00010000:0.0:1789760791.875019:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80437248 -> 80441344 00002000:00010000:0.0:1789760791.875024:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157128 -> 157136 00010000:00010000:0.0:1789760791.875037:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.875051:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.875060:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.875067:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.875072:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.875081:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.875100:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.875107:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2103 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80437248->80441343] (req 80437248->80441343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.875233:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6617us (6875us total) trans 4294986945 rc 0/0 00000100:00100000:0.0:1789760791.875244:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19649 00000100:00100000:0.0:1789760791.875253:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721980524803 time 721993207398 state 0x2 ref 0 00000100:00100000:1.0:1789760791.878802:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.879015:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503903360 00000100:00100000:0.0:1789760791.879026:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503903360 00000100:00100000:0.0:1789760791.879079:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721980524803 time 721993207398 state 0x0 ref 1 00000100:00100000:0.0:1789760791.879093:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721993467814 time 721993207398 state 0x1 ref 1 00000100:00100000:0.0:1789760791.879123:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721993467814 time 722003729712 state 0x0 ref 1 00000100:00100000:0.0:1789760791.879135:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.879185:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80441344, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265d0 00010000:00010000:0.0:1789760791.879193:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879209:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760791.879214:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.879220:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879246:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.884652:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986946 is committed 00002000:00010000:1.0:1789760791.884983:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80441344 -> 80445440 00002000:00010000:1.0:1789760791.884989:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157136 -> 157144 00010000:00010000:1.0:1789760791.885000:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.885014:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.885024:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.885031:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.885036:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.885044:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.885063:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.885071:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f210a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80441344->80445439] (req 80441344->80445439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.885207:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6068us (6419us total) trans 4294986946 rc 0/0 00000100:00100000:1.0:1789760791.885227:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19650 00000100:00100000:1.0:1789760791.885237:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721993467814 time 722003729712 state 0x2 ref 0 00000100:00100000:2.0:1789760791.890965:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.891074:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503903488 00000100:00100000:1.0:1789760791.891084:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503903488 00000100:00100000:1.0:1789760791.891135:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 721993467814 time 722003729712 state 0x0 ref 1 00000100:00100000:1.0:1789760791.891150:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722003990128 time 722003729712 state 0x1 ref 1 00000100:00100000:1.0:1789760791.891210:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722003990128 time 722015816536 state 0x0 ref 1 00000100:00100000:1.0:1789760791.891234:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.891276:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80445440, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265d8 00010000:00010000:1.0:1789760791.891284:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.891299:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760791.891304:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.891309:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.891323:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.898456:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986947 is committed 00002000:00010000:1.0:1789760791.898696:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80445440 -> 80449536 00002000:00010000:1.0:1789760791.898703:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157144 -> 157152 00010000:00010000:1.0:1789760791.898716:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.898728:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.898737:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.898750:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.898755:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.898763:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.898781:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.898789:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2111 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80445440->80449535] (req 80445440->80449535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.898880:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7644us (7919us total) trans 4294986947 rc 0/0 00000100:00100000:1.0:1789760791.898891:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19651 00000100:00100000:1.0:1789760791.898902:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722003990128 time 722015816536 state 0x2 ref 0 00000100:00100000:2.0:1789760791.903736:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.903886:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503903616 00000100:00100000:0.0:1789760791.903896:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503903616 00000100:00100000:0.0:1789760791.903950:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722003990128 time 722015816536 state 0x0 ref 1 00000100:00100000:0.0:1789760791.903965:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722016076952 time 722015816536 state 0x1 ref 1 00000100:00100000:0.0:1789760791.903990:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722016076952 time 722028596757 state 0x0 ref 1 00000100:00100000:0.0:1789760791.904001:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503903616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.904050:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80449536, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265e0 00010000:00010000:0.0:1789760791.904059:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.904075:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760791.904079:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.904086:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.904100:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.919879:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986948 is committed 00002000:00010000:0.0:1789760791.921617:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80449536 -> 80453632 00002000:00010000:0.0:1789760791.921626:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157152 -> 157160 00010000:00010000:0.0:1789760791.921642:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.921656:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.921671:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.921677:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.921682:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.921691:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.921710:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.921717:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2118 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80449536->80453631] (req 80449536->80453631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.921813:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503903616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 17808us (18080us total) trans 4294986948 rc 0/0 00000100:00100000:0.0:1789760791.921823:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19652 00000100:00100000:0.0:1789760791.921831:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722016076952 time 722028596757 state 0x2 ref 0 00000100:00100000:3.0:1789760791.926801:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.926903:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503903744 00000100:00100000:0.0:1789760791.926915:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503903744 00000100:00100000:0.0:1789760791.926973:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722016076952 time 722028596757 state 0x0 ref 1 00000100:00100000:0.0:1789760791.926988:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722028857173 time 722028596757 state 0x1 ref 1 00000100:00100000:0.0:1789760791.927021:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722028857173 time 722051626852 state 0x0 ref 1 00000100:00100000:0.0:1789760791.927032:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.927076:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80453632, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265e8 00010000:00010000:0.0:1789760791.927084:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.927100:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760791.927105:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.927110:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.927124:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.932507:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986949 is committed 00002000:00010000:0.0:1789760791.934198:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80453632 -> 80457728 00002000:00010000:0.0:1789760791.934207:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157160 -> 157168 00010000:00010000:0.0:1789760791.934358:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.934372:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.934380:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.934549:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.934619:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.934629:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.934649:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.934656:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f211f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80453632->80457727] (req 80453632->80457727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.934755:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7716us (7995us total) trans 4294986949 rc 0/0 00000100:00100000:0.0:1789760791.934766:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19653 00000100:00100000:0.0:1789760791.934778:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722028857173 time 722051626852 state 0x2 ref 0 00000100:00100000:2.0:1789760791.938648:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.939020:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503903872 00000100:00100000:1.0:1789760791.939029:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503903872 00000100:00100000:1.0:1789760791.939078:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722028857173 time 722051626852 state 0x0 ref 1 00000100:00100000:1.0:1789760791.939093:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722051887268 time 722051626852 state 0x1 ref 1 00000100:00100000:1.0:1789760791.939154:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722051887268 time 722063760129 state 0x0 ref 1 00000100:00100000:1.0:1789760791.939169:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.939237:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80457728, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265f0 00010000:00010000:1.0:1789760791.939247:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.939263:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760791.939267:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.939274:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.939287:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.949605:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986950 is committed 00002000:00010000:0.0:1789760791.949716:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80457728 -> 80461824 00002000:00010000:0.0:1789760791.949721:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157168 -> 157176 00010000:00010000:0.0:1789760791.949733:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.949745:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.949759:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.949766:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760791.949771:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.949780:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.949799:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.950635:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2126 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80457728->80461823] (req 80457728->80461823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.952184:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503903872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13006us (13536us total) trans 4294986950 rc 0/0 00000100:00100000:1.0:1789760791.952196:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19654 00000100:00100000:1.0:1789760791.952206:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722051887268 time 722063760129 state 0x2 ref 0 00000100:00100000:0.0:1789760791.956761:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760791.957186:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904000 00000100:00100000:1.0:1789760791.957196:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904000 00000100:00100000:1.0:1789760791.957319:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722051887268 time 722063760129 state 0x0 ref 1 00000100:00100000:1.0:1789760791.957334:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722064020545 time 722063760129 state 0x1 ref 1 00000100:00100000:1.0:1789760791.957427:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722064020545 time 722082033719 state 0x0 ref 1 00000100:00100000:1.0:1789760791.957444:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760791.957502:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80461824, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x265f8 00010000:00010000:1.0:1789760791.957512:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.957528:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760791.957540:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.957545:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.957559:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.963418:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986951 is committed 00002000:00010000:1.0:1789760791.963667:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80461824 -> 80465920 00002000:00010000:1.0:1789760791.963673:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157176 -> 157184 00010000:00010000:1.0:1789760791.963685:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.963698:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.963707:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.963714:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.963719:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.963728:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.963746:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.963754:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f212d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80461824->80465919] (req 80461824->80465919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.963914:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6467us (7158us total) trans 4294986951 rc 0/0 00000100:00100000:1.0:1789760791.963924:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19655 00000100:00100000:1.0:1789760791.963935:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722064020545 time 722082033719 state 0x2 ref 0 00000100:00100000:0.0:1789760791.967038:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.967627:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904128 00000100:00100000:2.0:1789760791.967638:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904128 00000100:00100000:2.0:1789760791.967694:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722064020545 time 722082033719 state 0x0 ref 1 00000100:00100000:2.0:1789760791.967713:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722082294135 time 722082033719 state 0x1 ref 1 00000100:00100000:2.0:1789760791.967737:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722082294135 time 722092344461 state 0x0 ref 1 00000100:00100000:2.0:1789760791.967750:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.967795:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80465920, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26600 00010000:00010000:2.0:1789760791.967805:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.967821:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760791.967826:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.967832:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.967845:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.972737:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986952 is committed 00002000:00010000:2.0:1789760791.972865:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80465920 -> 80470016 00002000:00010000:2.0:1789760791.972871:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157184 -> 157192 00010000:00010000:2.0:1789760791.972883:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.972947:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.972956:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.972965:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760791.972970:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.972980:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.972998:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.973005:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2134 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80465920->80470015] (req 80465920->80470015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.973143:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5389us (6108us total) trans 4294986952 rc 0/0 00000100:00100000:2.0:1789760791.973153:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19656 00000100:00100000:2.0:1789760791.973162:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722082294135 time 722092344461 state 0x2 ref 0 00000100:00100000:2.0:1789760791.977362:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760791.977480:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904256 00000100:00100000:0.0:1789760791.977500:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904256 00000100:00100000:0.0:1789760791.977556:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722082294135 time 722092344461 state 0x0 ref 1 00000100:00100000:0.0:1789760791.977570:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722092604877 time 722092344461 state 0x1 ref 1 00000100:00100000:0.0:1789760791.977600:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722092604877 time 722102206978 state 0x0 ref 1 00000100:00100000:0.0:1789760791.977615:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503904256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760791.977662:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80470016, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26608 00010000:00010000:0.0:1789760791.977672:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.977687:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760791.977691:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.977698:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.977712:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.984606:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986953 is committed 00002000:00010000:1.0:1789760791.984949:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80470016 -> 80474112 00002000:00010000:1.0:1789760791.984955:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157192 -> 157200 00010000:00010000:1.0:1789760791.984968:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.984981:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.984990:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.985003:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.985008:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.985017:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.985037:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.985044:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f213b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80470016->80474111] (req 80470016->80474111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.985230:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503904256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7605us (7862us total) trans 4294986953 rc 0/0 00000100:00100000:1.0:1789760791.985241:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19657 00000100:00100000:1.0:1789760791.985251:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722092604877 time 722102206978 state 0x2 ref 0 00000100:00100000:1.0:1789760791.990923:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760791.991190:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904384 00000100:00100000:2.0:1789760791.991199:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904384 00000100:00100000:2.0:1789760791.991252:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722092604877 time 722102206978 state 0x0 ref 1 00000100:00100000:2.0:1789760791.991266:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722102467394 time 722102206978 state 0x1 ref 1 00000100:00100000:2.0:1789760791.991371:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722102467394 time 722115977514 state 0x0 ref 1 00000100:00100000:2.0:1789760791.991385:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760791.991452:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80474112, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26610 00010000:00010000:2.0:1789760791.991462:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.991478:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760791.991482:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.991488:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.991516:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.998379:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986954 is committed 00002000:00010000:1.0:1789760791.998721:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80474112 -> 80478208 00002000:00010000:1.0:1789760791.998728:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157200 -> 157208 00010000:00010000:1.0:1789760791.998743:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998757:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998766:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998774:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760791.998780:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998788:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998807:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998816:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2142 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80474112->80478207] (req 80474112->80478207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.998950:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7564us (8033us total) trans 4294986954 rc 0/0 00000100:00100000:1.0:1789760791.998961:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19658 00000100:00100000:1.0:1789760791.998970:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722102467394 time 722115977514 state 0x2 ref 0 00000100:00100000:0.0:1789760792.008509:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.009146:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904512 00000100:00100000:1.0:1789760792.009157:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904512 00000100:00100000:1.0:1789760792.009289:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722102467394 time 722115977514 state 0x0 ref 1 00000100:00100000:1.0:1789760792.009306:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722116237930 time 722115977514 state 0x1 ref 1 00000100:00100000:1.0:1789760792.009339:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722116237930 time 722133945784 state 0x0 ref 1 00000100:00100000:1.0:1789760792.009352:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503904512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.009417:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80478208, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26618 00010000:00010000:1.0:1789760792.009426:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.009441:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.009446:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.009451:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.009466:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760792.013869:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760792.014144:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986955 is committed 00002000:00010000:1.0:1789760792.014286:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80478208 -> 80482304 00002000:00010000:1.0:1789760792.014292:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157208 -> 157216 00010000:00010000:1.0:1789760792.014304:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.014316:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.014324:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.014336:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.014340:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.014348:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.014364:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.014370:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2149 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80478208->80482303] (req 80478208->80482303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.014622:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503904512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5267us (6127us total) trans 4294986955 rc 0/0 00000100:00100000:1.0:1789760792.014633:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19659 00000100:00100000:1.0:1789760792.014643:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722116237930 time 722133945784 state 0x2 ref 0 00000100:00100000:0.0:1789760792.028614:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.028687:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904640 00000100:00100000:2.0:1789760792.028697:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904640 00000100:00100000:2.0:1789760792.028753:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722116237930 time 722133945784 state 0x0 ref 1 00000100:00100000:2.0:1789760792.028767:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722134206200 time 722133945784 state 0x1 ref 1 00000100:00100000:2.0:1789760792.028800:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722134206200 time 722153406788 state 0x0 ref 1 00000100:00100000:2.0:1789760792.028813:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.028858:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80482304, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26620 00010000:00010000:2.0:1789760792.028868:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.028884:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.028888:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.028894:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.028908:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.034387:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986956 is committed 00002000:00010000:2.0:1789760792.042356:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80482304 -> 80486400 00002000:00010000:2.0:1789760792.042365:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157216 -> 157224 00010000:00010000:2.0:1789760792.042378:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.042418:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.042427:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.042435:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.042440:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.042449:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.042470:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.042478:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2150 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80482304->80486399] (req 80482304->80486399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.042724:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13907us (14113us total) trans 4294986956 rc 0/0 00000100:00100000:2.0:1789760792.042734:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19660 00000100:00100000:2.0:1789760792.042743:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722134206200 time 722153406788 state 0x2 ref 0 00000100:00100000:2.0:1789760792.050153:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.050359:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904768 00000100:00100000:0.0:1789760792.050368:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904768 00000100:00100000:0.0:1789760792.050450:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722134206200 time 722153406788 state 0x0 ref 1 00000100:00100000:0.0:1789760792.050464:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722153667204 time 722153406788 state 0x1 ref 1 00000100:00100000:0.0:1789760792.050483:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722153667204 time 722175090928 state 0x0 ref 1 00000100:00100000:0.0:1789760792.050506:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.050616:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80486400, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26628 00010000:00010000:0.0:1789760792.050627:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.050641:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.050646:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.050652:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.050667:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.056137:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986957 is committed 00002000:00010000:0.0:1789760792.056270:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80486400 -> 80490496 00002000:00010000:0.0:1789760792.056276:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157224 -> 157232 00010000:00010000:0.0:1789760792.056288:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.056299:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.056307:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.056313:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.056318:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.056327:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.056347:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.056355:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2157 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80486400->80490495] (req 80486400->80490495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.056689:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6180us (6539us total) trans 4294986957 rc 0/0 00000100:00100000:0.0:1789760792.056699:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19661 00000100:00100000:0.0:1789760792.056708:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722153667204 time 722175090928 state 0x2 ref 0 00000100:00100000:0.0:1789760792.065979:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.066101:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503904896 00000100:00100000:1.0:1789760792.066110:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503904896 00000100:00100000:1.0:1789760792.066163:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722153667204 time 722175090928 state 0x0 ref 1 00000100:00100000:1.0:1789760792.066178:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722175351344 time 722175090928 state 0x1 ref 1 00000100:00100000:1.0:1789760792.066262:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722175351344 time 722190867629 state 0x0 ref 1 00000100:00100000:1.0:1789760792.066279:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.066327:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80490496, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26630 00010000:00010000:1.0:1789760792.066336:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.066354:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.066359:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.066364:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.066379:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.070560:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986958 is committed 00002000:00010000:1.0:1789760792.070674:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80490496 -> 80494592 00002000:00010000:1.0:1789760792.070679:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157232 -> 157240 00010000:00010000:1.0:1789760792.070693:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.070704:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.070713:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.070719:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.070724:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.070734:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.070752:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.070761:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f215e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80490496->80494591] (req 80490496->80494591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.070854:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503904896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4573us (4880us total) trans 4294986958 rc 0/0 00000100:00100000:1.0:1789760792.070865:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19662 00000100:00100000:1.0:1789760792.070874:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722175351344 time 722190867629 state 0x2 ref 0 00000100:00100000:2.0:1789760792.082567:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.082663:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905024 00000100:00100000:1.0:1789760792.082673:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905024 00000100:00100000:1.0:1789760792.082730:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722175351344 time 722190867629 state 0x0 ref 1 00000100:00100000:1.0:1789760792.082746:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722191128045 time 722190867629 state 0x1 ref 1 00000100:00100000:1.0:1789760792.082778:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722191128045 time 722207384278 state 0x0 ref 1 00000100:00100000:1.0:1789760792.082790:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.082835:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80494592, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26638 00010000:00010000:1.0:1789760792.082845:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.082860:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.082865:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.082872:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.082885:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.083319:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760792.097812:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986959 is committed 00002000:00010000:1.0:1789760792.097917:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80494592 -> 80498688 00002000:00010000:1.0:1789760792.097921:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157240 -> 157248 00010000:00010000:1.0:1789760792.097933:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.097944:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.097954:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.097963:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.097968:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.097979:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.098000:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.098008:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2165 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80494592->80498687] (req 80494592->80498687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.098098:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15305us (15535us total) trans 4294986959 rc 0/0 00000100:00100000:1.0:1789760792.098109:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19663 00000100:00100000:1.0:1789760792.098118:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722191128045 time 722207384278 state 0x2 ref 0 00000100:00100000:2.0:1789760792.103385:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.103546:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905152 00000100:00100000:0.0:1789760792.103557:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905152 00000100:00100000:0.0:1789760792.103611:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722191128045 time 722207384278 state 0x0 ref 1 00000100:00100000:0.0:1789760792.103627:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722207644694 time 722207384278 state 0x1 ref 1 00000100:00100000:0.0:1789760792.103689:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722207644694 time 722228294700 state 0x0 ref 1 00000100:00100000:0.0:1789760792.103703:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.103749:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80498688, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26640 00010000:00010000:0.0:1789760792.103759:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.103781:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.103786:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.103792:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.103805:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.110449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986960 is committed 00002000:00010000:0.0:1789760792.110653:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80498688 -> 80502784 00002000:00010000:0.0:1789760792.110658:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157248 -> 157256 00010000:00010000:0.0:1789760792.110671:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.110685:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.110694:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.110702:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.110708:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.110717:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.110741:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.110749:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f216c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80498688->80502783] (req 80498688->80502783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.110842:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7136us (7462us total) trans 4294986960 rc 0/0 00000100:00100000:0.0:1789760792.110853:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19664 00000100:00100000:0.0:1789760792.110864:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722207644694 time 722228294700 state 0x2 ref 0 00000100:00100000:0.0:1789760792.116875:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.117238:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905280 00000100:00100000:1.0:1789760792.117247:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905280 00000100:00100000:1.0:1789760792.117300:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722207644694 time 722228294700 state 0x0 ref 1 00000100:00100000:1.0:1789760792.117316:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722228555116 time 722228294700 state 0x1 ref 1 00000100:00100000:1.0:1789760792.117425:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722228555116 time 722241997726 state 0x0 ref 1 00000100:00100000:1.0:1789760792.117439:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.117486:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80502784, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26648 00010000:00010000:1.0:1789760792.117510:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.117526:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.117531:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.117548:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.117563:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.127303:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986961 is committed 00002000:00010000:1.0:1789760792.127515:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80502784 -> 80506880 00002000:00010000:1.0:1789760792.127521:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157256 -> 157264 00010000:00010000:1.0:1789760792.127535:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.127546:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.127555:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.127563:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.127567:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.127577:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.127597:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.127608:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2173 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80502784->80506879] (req 80502784->80506879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.127698:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10257us (10827us total) trans 4294986961 rc 0/0 00000100:00100000:1.0:1789760792.127708:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19665 00000100:00100000:1.0:1789760792.127718:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722228555116 time 722241997726 state 0x2 ref 0 00000100:00100000:2.0:1789760792.142206:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760792.142408:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905408 00000100:00100000:3.0:1789760792.142418:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905408 00000100:00100000:2.0:1789760792.143892:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722228555116 time 722241997726 state 0x0 ref 1 00000100:00100000:2.0:1789760792.143914:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722242258142 time 722241997726 state 0x1 ref 1 00000100:00100000:2.0:1789760792.143996:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722242258142 time 722268601396 state 0x0 ref 1 00000100:00100000:2.0:1789760792.144010:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.144055:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80506880, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26650 00010000:00010000:2.0:1789760792.144063:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.144079:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.144083:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.144089:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.144102:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.150552:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986962 is committed 00002000:00010000:2.0:1789760792.150786:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80506880 -> 80510976 00002000:00010000:2.0:1789760792.150792:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157264 -> 157272 00010000:00010000:2.0:1789760792.150805:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.150817:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.150825:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.150833:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.150838:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.150846:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.150870:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.150878:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f217a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80506880->80510975] (req 80506880->80510975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.150969:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6958us (8769us total) trans 4294986962 rc 0/0 00000100:00100000:2.0:1789760792.150981:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19666 00000100:00100000:2.0:1789760792.150991:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722242258142 time 722268601396 state 0x2 ref 0 00000100:00100000:2.0:1789760792.156963:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.157115:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905536 00000100:00100000:1.0:1789760792.157125:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905536 00000100:00100000:1.0:1789760792.157181:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722242258142 time 722268601396 state 0x0 ref 1 00000100:00100000:1.0:1789760792.157197:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722268861812 time 722268601396 state 0x1 ref 1 00000100:00100000:1.0:1789760792.157231:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722268861812 time 722281837747 state 0x0 ref 1 00000100:00100000:1.0:1789760792.157243:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.157287:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80510976, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26658 00010000:00010000:1.0:1789760792.157296:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.157312:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.157317:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.157323:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.157343:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.163059:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986963 is committed 00002000:00010000:1.0:1789760792.163180:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80510976 -> 80515072 00002000:00010000:1.0:1789760792.163186:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157272 -> 157280 00010000:00010000:1.0:1789760792.163198:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.163221:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.163231:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.163239:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.163247:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.163256:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.163275:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.163282:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2181 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80510976->80515071] (req 80510976->80515071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.163585:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6338us (6626us total) trans 4294986963 rc 0/0 00000100:00100000:1.0:1789760792.163596:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19667 00000100:00100000:1.0:1789760792.163606:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722268861812 time 722281837747 state 0x2 ref 0 00000100:00100000:0.0:1789760792.167129:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.167463:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905664 00000100:00100000:2.0:1789760792.167473:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905664 00000100:00100000:2.0:1789760792.167688:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722268861812 time 722281837747 state 0x0 ref 1 00000100:00100000:2.0:1789760792.167704:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722282098163 time 722281837747 state 0x1 ref 1 00000100:00100000:2.0:1789760792.167769:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722282098163 time 722292375057 state 0x0 ref 1 00000100:00100000:2.0:1789760792.167791:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.167838:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80515072, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26660 00010000:00010000:2.0:1789760792.167847:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.167867:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.167872:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.167879:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.167893:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.174944:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986964 is committed 00002000:00010000:2.0:1789760792.179209:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80515072 -> 80519168 00002000:00010000:2.0:1789760792.179218:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157280 -> 157288 00010000:00010000:2.0:1789760792.179232:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.179246:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.179255:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.179261:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.179265:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.179274:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.179299:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.179307:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2188 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80515072->80519167] (req 80515072->80519167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.179445:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11660us (12319us total) trans 4294986964 rc 0/0 00000100:00100000:2.0:1789760792.179456:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19668 00000100:00100000:2.0:1789760792.179465:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722282098163 time 722292375057 state 0x2 ref 0 00000100:00100000:2.0:1789760792.184275:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.184428:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905792 00000100:00100000:0.0:1789760792.184439:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905792 00000100:00100000:0.0:1789760792.184501:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722282098163 time 722292375057 state 0x0 ref 1 00000100:00100000:0.0:1789760792.184518:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722292635473 time 722292375057 state 0x1 ref 1 00000100:00100000:0.0:1789760792.184546:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722292635473 time 722309152959 state 0x0 ref 1 00000100:00100000:0.0:1789760792.184559:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.184604:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80519168, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26668 00010000:00010000:0.0:1789760792.184613:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.184633:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.184638:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.184644:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.184658:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.189361:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986965 is committed 00002000:00010000:0.0:1789760792.190766:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80519168 -> 80523264 00002000:00010000:0.0:1789760792.190774:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157288 -> 157296 00010000:00010000:0.0:1789760792.190787:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.190799:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.190809:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.190816:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.190820:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.190828:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.190854:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.190861:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f218f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80519168->80523263] (req 80519168->80523263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.191146:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503905792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6583us (6875us total) trans 4294986965 rc 0/0 00000100:00100000:0.0:1789760792.191156:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19669 00000100:00100000:0.0:1789760792.191166:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722292635473 time 722309152959 state 0x2 ref 0 00000100:00100000:1.0:1789760792.197976:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.198078:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503905920 00000100:00100000:2.0:1789760792.198088:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503905920 00000100:00100000:2.0:1789760792.198141:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722292635473 time 722309152959 state 0x0 ref 1 00000100:00100000:2.0:1789760792.198156:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722309413375 time 722309152959 state 0x1 ref 1 00000100:00100000:2.0:1789760792.198189:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722309413375 time 722322795844 state 0x0 ref 1 00000100:00100000:2.0:1789760792.198201:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.198249:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80523264, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26670 00010000:00010000:2.0:1789760792.198259:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.198282:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.198286:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.198292:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.198305:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.204577:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986966 is committed 00002000:00010000:2.0:1789760792.204773:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80523264 -> 80527360 00002000:00010000:2.0:1789760792.204779:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157296 -> 157304 00010000:00010000:2.0:1789760792.204792:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.204871:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.204881:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.205019:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.205025:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.205035:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.205061:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.205069:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2196 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80523264->80527359] (req 80523264->80527359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.205195:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503905920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6991us (7222us total) trans 4294986966 rc 0/0 00000100:00100000:2.0:1789760792.205206:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19670 00000100:00100000:2.0:1789760792.205226:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722309413375 time 722322795844 state 0x2 ref 0 00000100:00100000:2.0:1789760792.209557:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.209793:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906048 00000100:00100000:1.0:1789760792.209870:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906048 00000100:00100000:1.0:1789760792.209926:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722309413375 time 722322795844 state 0x0 ref 1 00000100:00100000:1.0:1789760792.209942:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722323056260 time 722322795844 state 0x1 ref 1 00000100:00100000:1.0:1789760792.209971:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722323056260 time 722334577914 state 0x0 ref 1 00000100:00100000:1.0:1789760792.209984:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.210044:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80527360, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26678 00010000:00010000:1.0:1789760792.210053:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.210069:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.210074:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.210080:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.210095:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.214848:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986967 is committed 00002000:00010000:1.0:1789760792.215078:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80527360 -> 80531456 00002000:00010000:1.0:1789760792.215084:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157304 -> 157312 00010000:00010000:1.0:1789760792.215097:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.215110:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.215118:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.215124:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.215128:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.215137:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.215155:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.215164:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f219d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80527360->80531455] (req 80527360->80531455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.215359:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5372us (5805us total) trans 4294986967 rc 0/0 00000100:00100000:1.0:1789760792.215371:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19671 00000100:00100000:1.0:1789760792.215381:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722323056260 time 722334577914 state 0x2 ref 0 00000100:00100000:0.0:1789760792.224730:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.224824:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906176 00000100:00100000:2.0:1789760792.224832:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906176 00000100:00100000:2.0:1789760792.224884:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722323056260 time 722334577914 state 0x0 ref 1 00000100:00100000:2.0:1789760792.224909:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722334838330 time 722334577914 state 0x1 ref 1 00000100:00100000:2.0:1789760792.224936:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722334838330 time 722349542953 state 0x0 ref 1 00000100:00100000:2.0:1789760792.224948:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503906176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.225159:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80531456, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26680 00010000:00010000:2.0:1789760792.225168:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.225189:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.225194:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.225199:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.225212:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.229792:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986968 is committed 00002000:00010000:2.0:1789760792.229927:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80531456 -> 80535552 00002000:00010000:2.0:1789760792.229932:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157312 -> 157320 00010000:00010000:2.0:1789760792.229945:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.229967:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.229976:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.229983:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.229988:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.229996:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.230020:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.230028:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80531456->80535551] (req 80531456->80535551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.230121:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503906176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5170us (5395us total) trans 4294986968 rc 0/0 00000100:00100000:2.0:1789760792.230131:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19672 00000100:00100000:2.0:1789760792.230140:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722334838330 time 722349542953 state 0x2 ref 0 00000100:00100000:2.0:1789760792.235215:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.235451:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906304 00000100:00100000:0.0:1789760792.235462:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906304 00000100:00100000:0.0:1789760792.235532:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722334838330 time 722349542953 state 0x0 ref 1 00000100:00100000:0.0:1789760792.235547:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722349803369 time 722349542953 state 0x1 ref 1 00000100:00100000:0.0:1789760792.235632:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722349803369 time 722360236843 state 0x0 ref 1 00000100:00100000:0.0:1789760792.235646:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.235696:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80535552, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26688 00010000:00010000:0.0:1789760792.235706:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.235721:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.235725:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.235731:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.235745:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.248072:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986969 is committed 00002000:00010000:0.0:1789760792.248463:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80535552 -> 80539648 00002000:00010000:0.0:1789760792.248469:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157320 -> 157328 00010000:00010000:0.0:1789760792.248482:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.248503:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.248513:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.248521:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.248526:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.248533:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.248552:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.248558:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f21ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80535552->80539647] (req 80535552->80539647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.248640:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12993us (13430us total) trans 4294986969 rc 0/0 00000100:00100000:0.0:1789760792.248651:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19673 00000100:00100000:0.0:1789760792.248661:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722349803369 time 722360236843 state 0x2 ref 0 00000100:00100000:0.0:1789760792.254953:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.256093:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906432 00000100:00100000:1.0:1789760792.256105:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906432 00000100:00100000:1.0:1789760792.256157:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722349803369 time 722360236843 state 0x0 ref 1 00000100:00100000:1.0:1789760792.256172:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722360497259 time 722360236843 state 0x1 ref 1 00000100:00100000:1.0:1789760792.256197:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722360497259 time 722380803999 state 0x0 ref 1 00000100:00100000:1.0:1789760792.256209:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc13450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503906432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.256253:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80539648, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26690 00010000:00010000:1.0:1789760792.256262:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.256276:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.256281:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.256287:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.256300:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.265614:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986970 is committed 00002000:00010000:1.0:1789760792.265725:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80539648 -> 80543744 00002000:00010000:1.0:1789760792.265730:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157328 -> 157336 00010000:00010000:1.0:1789760792.265743:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.265756:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.265765:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.265773:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.265777:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.265786:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.265805:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.265813:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80539648->80543743] (req 80539648->80543743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.265898:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc13450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503906432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9687us (10949us total) trans 4294986970 rc 0/0 00000100:00100000:1.0:1789760792.265907:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19674 00000100:00100000:1.0:1789760792.265915:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722360497259 time 722380803999 state 0x2 ref 0 00000100:00100000:2.0:1789760792.271612:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.271687:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906560 00000100:00100000:1.0:1789760792.271697:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906560 00000100:00100000:1.0:1789760792.271747:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722360497259 time 722380803999 state 0x0 ref 1 00000100:00100000:1.0:1789760792.271763:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722381064415 time 722380803999 state 0x1 ref 1 00000100:00100000:1.0:1789760792.271795:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722381064415 time 722396401135 state 0x0 ref 1 00000100:00100000:1.0:1789760792.271807:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.271852:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80543744, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26698 00010000:00010000:1.0:1789760792.271860:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.271875:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.271879:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.271885:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.271898:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.277008:0:3639:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760792.277031:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523615104:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760792.277111:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.277236:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523615104 00000100:00100000:2.0:1789760792.277244:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523615104 00000100:00100000:2.0:1789760792.277290:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1789760792.277299:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3639:x1876699523615104:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789760792.277330:0:7941:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:2.0:1789760792.277409:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3639:x1876699523615104:12345-0@lo:41:osp-pre-0-1.0 Request processed in 93us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760792.277415:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:0.0:1789760792.277510:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523615104:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789760792.279597:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986971 is committed 00002000:00010000:2.0:1789760792.279819:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80543744 -> 80547840 00002000:00010000:2.0:1789760792.279824:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157336 -> 157344 00010000:00010000:2.0:1789760792.279837:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.279849:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.279858:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.279865:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.279869:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.279878:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.279896:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.279905:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f21b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80543744->80547839] (req 80543744->80547839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.280043:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8234us (8436us total) trans 4294986971 rc 0/0 00000100:00100000:2.0:1789760792.280054:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19675 00000100:00100000:2.0:1789760792.280063:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722381064415 time 722396401135 state 0x2 ref 0 00000100:00100000:2.0:1789760792.283376:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.283671:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906688 00000100:00100000:0.0:1789760792.283681:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906688 00000100:00100000:0.0:1789760792.283733:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722381064415 time 722396401135 state 0x0 ref 1 00000100:00100000:0.0:1789760792.283744:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722396661551 time 722396401135 state 0x1 ref 1 00000100:00100000:0.0:1789760792.283794:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722396661551 time 722408401720 state 0x0 ref 1 00000100:00100000:0.0:1789760792.283804:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503906688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.283843:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80547840, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266a0 00010000:00010000:0.0:1789760792.283850:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.283863:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.283866:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.283870:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.283880:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.290872:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986972 is committed 00002000:00010000:0.0:1789760792.291061:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80547840 -> 80551936 00002000:00010000:0.0:1789760792.291066:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157344 -> 157352 00010000:00010000:0.0:1789760792.291079:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.291160:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.291171:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.291178:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.291183:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.291197:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.291227:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.291235:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80547840->80551935] (req 80547840->80551935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.291723:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503906688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7915us (8352us total) trans 4294986972 rc 0/0 00000100:00100000:0.0:1789760792.291733:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19676 00000100:00100000:0.0:1789760792.291742:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722396661551 time 722408401720 state 0x2 ref 0 00000100:00100000:0.0:1789760792.295843:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.296726:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906816 00000100:00100000:1.0:1789760792.296738:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906816 00000100:00100000:0.0:1789760792.296866:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722396661551 time 722408401720 state 0x0 ref 1 00000100:00100000:0.0:1789760792.296882:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722408662136 time 722408401720 state 0x1 ref 1 00000100:00100000:0.0:1789760792.296913:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722408662136 time 722421520231 state 0x0 ref 1 00000100:00100000:0.0:1789760792.297036:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503906816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.297459:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80551936, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266a8 00010000:00010000:0.0:1789760792.297470:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.297486:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.297501:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.297507:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.297522:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.305459:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986973 is committed 00002000:00010000:0.0:1789760792.305588:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80551936 -> 80556032 00002000:00010000:0.0:1789760792.305593:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157352 -> 157360 00010000:00010000:0.0:1789760792.305605:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.305618:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.305627:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.305633:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.305638:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.305647:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.305670:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.305677:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f21c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80551936->80556031] (req 80551936->80556031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.305770:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503906816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8841us (9931us total) trans 4294986973 rc 0/0 00000100:00100000:0.0:1789760792.305779:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19677 00000100:00100000:0.0:1789760792.305788:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722408662136 time 722421520231 state 0x2 ref 0 00000100:00100000:2.0:1789760792.308620:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.308716:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503906944 00000100:00100000:1.0:1789760792.308726:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503906944 00000100:00100000:1.0:1789760792.308783:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722408662136 time 722421520231 state 0x0 ref 1 00000100:00100000:1.0:1789760792.308802:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722421780647 time 722421520231 state 0x1 ref 1 00000100:00100000:1.0:1789760792.308833:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722421780647 time 722433439403 state 0x0 ref 1 00000100:00100000:1.0:1789760792.308844:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.308890:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80556032, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266b0 00010000:00010000:1.0:1789760792.308898:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.308914:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.308918:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.308924:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.308938:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.319145:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986974 is committed 00002000:00010000:1.0:1789760792.319776:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80556032 -> 80560128 00002000:00010000:1.0:1789760792.319782:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157360 -> 157368 00010000:00010000:1.0:1789760792.319795:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.319807:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.319815:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.319822:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.319826:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.319834:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.319851:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.319859:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f21ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80556032->80560127] (req 80556032->80560127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.319959:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503906944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11111us (11343us total) trans 4294986974 rc 0/0 00000100:00100000:1.0:1789760792.319968:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19678 00000100:00100000:1.0:1789760792.319978:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722421780647 time 722433439403 state 0x2 ref 0 00000100:00100000:2.0:1789760792.324106:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.324163:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907072 00000100:00100000:1.0:1789760792.324173:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907072 00000100:00100000:1.0:1789760792.324231:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722421780647 time 722433439403 state 0x0 ref 1 00000100:00100000:1.0:1789760792.324246:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722433699819 time 722433439403 state 0x1 ref 1 00000100:00100000:1.0:1789760792.324280:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722433699819 time 722448886270 state 0x0 ref 1 00000100:00100000:1.0:1789760792.324298:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503907072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.324345:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80560128, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266b8 00010000:00010000:1.0:1789760792.324354:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.324370:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.324375:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.324381:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.324425:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.337951:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986975 is committed 00002000:00010000:1.0:1789760792.338114:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80560128 -> 80564224 00002000:00010000:1.0:1789760792.338120:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157368 -> 157376 00010000:00010000:1.0:1789760792.338133:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.338147:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.338157:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.338165:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.338170:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.338179:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.338200:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.338209:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f21d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80560128->80564223] (req 80560128->80564223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.338306:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503907072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14006us (14204us total) trans 4294986975 rc 0/0 00000100:00100000:1.0:1789760792.338318:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19679 00000100:00100000:1.0:1789760792.338328:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722433699819 time 722448886270 state 0x2 ref 0 00000100:00100000:0.0:1789760792.341791:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.341891:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907200 00000100:00100000:2.0:1789760792.341901:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907200 00000100:00100000:2.0:1789760792.341957:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722433699819 time 722448886270 state 0x0 ref 1 00000100:00100000:2.0:1789760792.341972:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722449146686 time 722448886270 state 0x1 ref 1 00000100:00100000:2.0:1789760792.342017:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722449146686 time 722466622929 state 0x0 ref 1 00000100:00100000:2.0:1789760792.342030:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.342079:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80564224, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266c0 00010000:00010000:2.0:1789760792.342088:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.342104:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.342109:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.342115:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.342129:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.348770:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986976 is committed 00002000:00010000:2.0:1789760792.348897:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80564224 -> 80568320 00002000:00010000:2.0:1789760792.348902:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157376 -> 157384 00010000:00010000:2.0:1789760792.348916:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.348935:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.348945:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.348952:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.348956:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.348965:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.348984:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.348992:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80564224->80568319] (req 80564224->80568319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.349083:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7050us (7295us total) trans 4294986976 rc 0/0 00000100:00100000:2.0:1789760792.349094:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19680 00000100:00100000:2.0:1789760792.349103:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722449146686 time 722466622929 state 0x2 ref 0 00000100:00100000:2.0:1789760792.356384:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.357720:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907328 00000100:00100000:0.0:1789760792.357733:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907328 00000100:00100000:0.0:1789760792.357793:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722449146686 time 722466622929 state 0x0 ref 1 00000100:00100000:0.0:1789760792.357808:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722466883345 time 722466622929 state 0x1 ref 1 00000100:00100000:0.0:1789760792.357841:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722466883345 time 722482447429 state 0x0 ref 1 00000100:00100000:0.0:1789760792.357854:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.357899:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80568320, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266c8 00010000:00010000:0.0:1789760792.357909:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.357923:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.357927:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.357933:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.357947:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.364803:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986977 is committed 00002000:00010000:1.0:1789760792.368636:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80568320 -> 80572416 00002000:00010000:1.0:1789760792.368646:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157384 -> 157392 00010000:00010000:1.0:1789760792.368659:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.368671:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.368680:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.368694:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.368699:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.368707:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.369061:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.369072:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f21e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80568320->80572415] (req 80568320->80572415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.369208:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11352us (12829us total) trans 4294986977 rc 0/0 00000100:00100000:1.0:1789760792.369219:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19681 00000100:00100000:1.0:1789760792.369229:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722466883345 time 722482447429 state 0x2 ref 0 00000100:00100000:1.0:1789760792.375310:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.375428:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907456 00000100:00100000:2.0:1789760792.375438:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907456 00000100:00100000:2.0:1789760792.375502:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722466883345 time 722482447429 state 0x0 ref 1 00000100:00100000:2.0:1789760792.375517:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722482707845 time 722482447429 state 0x1 ref 1 00000100:00100000:2.0:1789760792.375548:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722482707845 time 722500155331 state 0x0 ref 1 00000100:00100000:2.0:1789760792.375560:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.375606:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80572416, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266d0 00010000:00010000:2.0:1789760792.375614:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.375630:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.375634:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.375641:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.375654:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.381427:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986978 is committed 00002000:00010000:1.0:1789760792.381721:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80572416 -> 80576512 00002000:00010000:1.0:1789760792.381728:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157392 -> 157400 00010000:00010000:1.0:1789760792.381743:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.381756:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.381765:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.381773:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.381777:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.381787:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.381807:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.384719:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f21ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80572416->80576511] (req 80572416->80576511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.384883:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9319us (9578us total) trans 4294986978 rc 0/0 00000100:00100000:1.0:1789760792.384894:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19682 00000100:00100000:1.0:1789760792.384905:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722482707845 time 722500155331 state 0x2 ref 0 00000100:00100000:1.0:1789760792.390070:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.390148:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907584 00000100:00100000:2.0:1789760792.390158:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907584 00000100:00100000:2.0:1789760792.390226:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722482707845 time 722500155331 state 0x0 ref 1 00000100:00100000:2.0:1789760792.390242:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722500415747 time 722500155331 state 0x1 ref 1 00000100:00100000:2.0:1789760792.390277:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722500415747 time 722514883150 state 0x0 ref 1 00000100:00100000:2.0:1789760792.390300:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.390349:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80576512, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266d8 00010000:00010000:2.0:1789760792.390358:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.390374:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.390379:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.390385:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.390438:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.398655:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986979 is committed 00002000:00010000:2.0:1789760792.399643:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80576512 -> 80580608 00002000:00010000:2.0:1789760792.399650:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157400 -> 157408 00010000:00010000:2.0:1789760792.399663:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.399675:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.399684:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.399695:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.399700:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.399708:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.399728:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.399736:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f21f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80576512->80580607] (req 80576512->80580607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.399850:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9557us (9783us total) trans 4294986979 rc 0/0 00000100:00100000:2.0:1789760792.399861:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19683 00000100:00100000:2.0:1789760792.399870:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722500415747 time 722514883150 state 0x2 ref 0 00000100:00100000:2.0:1789760792.404360:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.404730:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907712 00000100:00100000:0.0:1789760792.404740:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907712 00000100:00100000:0.0:1789760792.404793:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722500415747 time 722514883150 state 0x0 ref 1 00000100:00100000:0.0:1789760792.404807:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722515143566 time 722514883150 state 0x1 ref 1 00000100:00100000:0.0:1789760792.404839:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722515143566 time 722529445995 state 0x0 ref 1 00000100:00100000:0.0:1789760792.404852:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.404897:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80580608, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266e0 00010000:00010000:0.0:1789760792.404906:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.404922:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.404927:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.404934:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.404947:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.416694:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986980 is committed 00002000:00010000:1.0:1789760792.416820:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80580608 -> 80584704 00002000:00010000:1.0:1789760792.416825:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157408 -> 157416 00010000:00010000:1.0:1789760792.416839:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.416853:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.416862:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.416869:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.416873:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.416883:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.416905:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.416912:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f21f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80580608->80584703] (req 80580608->80584703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.417007:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12152us (12650us total) trans 4294986980 rc 0/0 00000100:00100000:1.0:1789760792.417017:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19684 00000100:00100000:1.0:1789760792.417028:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722515143566 time 722529445995 state 0x2 ref 0 00000100:00100000:1.0:1789760792.421812:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.422043:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907840 00000100:00100000:0.0:1789760792.422052:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907840 00000100:00100000:0.0:1789760792.422116:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722515143566 time 722529445995 state 0x0 ref 1 00000100:00100000:0.0:1789760792.422131:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722529706411 time 722529445995 state 0x1 ref 1 00000100:00100000:0.0:1789760792.422164:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722529706411 time 722546770946 state 0x0 ref 1 00000100:00100000:0.0:1789760792.422177:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.422230:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80584704, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266e8 00010000:00010000:0.0:1789760792.422239:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.422255:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.422259:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.422265:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.422278:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.428474:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986981 is committed 00002000:00010000:1.0:1789760792.428896:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80584704 -> 80588800 00002000:00010000:1.0:1789760792.428903:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157416 -> 157424 00010000:00010000:1.0:1789760792.428921:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.428937:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.428946:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.428954:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.428960:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.428970:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.428995:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.429004:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f21ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80584704->80588799] (req 80584704->80588799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.429170:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503907840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6987us (7359us total) trans 4294986981 rc 0/0 00000100:00100000:1.0:1789760792.429183:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19685 00000100:00100000:1.0:1789760792.429196:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722529706411 time 722546770946 state 0x2 ref 0 00000100:00100000:2.0:1789760792.437102:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.437183:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503907968 00000100:00100000:1.0:1789760792.437197:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503907968 00000100:00100000:1.0:1789760792.437275:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722529706411 time 722546770946 state 0x0 ref 1 00000100:00100000:1.0:1789760792.437294:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722547031362 time 722546770946 state 0x1 ref 1 00000100:00100000:1.0:1789760792.437334:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722547031362 time 722561939634 state 0x0 ref 1 00000100:00100000:1.0:1789760792.437348:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503907968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.437768:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80588800, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266f0 00010000:00010000:1.0:1789760792.437780:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.437801:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.437806:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.437813:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.437832:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.445098:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986982 is committed 00002000:00010000:1.0:1789760792.445267:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80588800 -> 80592896 00002000:00010000:1.0:1789760792.445275:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157424 -> 157432 00010000:00010000:1.0:1789760792.445292:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.445315:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.445326:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.445334:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.445340:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.445350:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.445377:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.445385:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2206 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80588800->80592895] (req 80588800->80592895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.445625:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503907968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8272us (8535us total) trans 4294986982 rc 0/0 00000100:00100000:1.0:1789760792.445639:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19686 00000100:00100000:1.0:1789760792.445652:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722547031362 time 722561939634 state 0x2 ref 0 00000100:00100000:2.0:1789760792.449289:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.449646:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908096 00000100:00100000:1.0:1789760792.449660:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908096 00000100:00100000:1.0:1789760792.449742:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722547031362 time 722561939634 state 0x0 ref 1 00000100:00100000:1.0:1789760792.449761:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722562200050 time 722561939634 state 0x1 ref 1 00000100:00100000:1.0:1789760792.449840:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722562200050 time 722574445994 state 0x0 ref 1 00000100:00100000:1.0:1789760792.449854:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.449916:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80592896, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x266f8 00010000:00010000:1.0:1789760792.449927:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.449947:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.449952:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.449959:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.449976:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.457030:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986983 is committed 00002000:00010000:1.0:1789760792.458071:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80592896 -> 80596992 00002000:00010000:1.0:1789760792.458081:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157432 -> 157440 00010000:00010000:1.0:1789760792.458098:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.458114:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.458130:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.458139:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.458145:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.458154:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.458179:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.458187:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f220d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80592896->80596991] (req 80592896->80596991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.458321:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8459us (9037us total) trans 4294986983 rc 0/0 00000100:00100000:1.0:1789760792.458334:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19687 00000100:00100000:1.0:1789760792.458348:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722562200050 time 722574445994 state 0x2 ref 0 00000100:00100000:2.0:1789760792.468045:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.468525:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908224 00000100:00100000:1.0:1789760792.468675:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908224 00000100:00100000:1.0:1789760792.468760:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722562200050 time 722574445994 state 0x0 ref 1 00000100:00100000:1.0:1789760792.468779:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722574706410 time 722574445994 state 0x1 ref 1 00000100:00100000:1.0:1789760792.468817:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722574706410 time 722593422769 state 0x0 ref 1 00000100:00100000:1.0:1789760792.468832:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503908224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.468904:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80596992, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26700 00010000:00010000:1.0:1789760792.468916:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.468938:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.468943:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.468951:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.468967:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.475136:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986984 is committed 00002000:00010000:1.0:1789760792.475296:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80596992 -> 80601088 00002000:00010000:1.0:1789760792.475302:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157440 -> 157448 00010000:00010000:1.0:1789760792.475319:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.475334:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.475344:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.475359:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.475365:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.475376:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.475531:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.475540:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2214 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80596992->80601087] (req 80596992->80601087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.475758:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503908224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6923us (7722us total) trans 4294986984 rc 0/0 00000100:00100000:1.0:1789760792.475771:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19688 00000100:00100000:1.0:1789760792.475783:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722574706410 time 722593422769 state 0x2 ref 0 00000100:00100000:2.0:1789760792.483223:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.483351:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908352 00000100:00100000:0.0:1789760792.483365:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908352 00000100:00100000:0.0:1789760792.483485:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722574706410 time 722593422769 state 0x0 ref 1 00000100:00100000:0.0:1789760792.483516:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722593683185 time 722593422769 state 0x1 ref 1 00000100:00100000:0.0:1789760792.483541:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722593683185 time 722608147846 state 0x0 ref 1 00000100:00100000:0.0:1789760792.483567:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.483634:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80601088, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26708 00010000:00010000:0.0:1789760792.483645:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.483663:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.483668:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.483675:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.483691:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.492993:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986985 is committed 00002000:00010000:0.0:1789760792.493145:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80601088 -> 80605184 00002000:00010000:0.0:1789760792.493151:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157448 -> 157456 00010000:00010000:0.0:1789760792.493168:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.493184:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.493193:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.493201:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.493207:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.493218:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.493243:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.493252:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f221b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80601088->80605183] (req 80601088->80605183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.493436:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9829us (10182us total) trans 4294986985 rc 0/0 00000100:00100000:0.0:1789760792.493450:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19689 00000100:00100000:0.0:1789760792.493462:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722593683185 time 722608147846 state 0x2 ref 0 00000100:00100000:2.0:1789760792.502920:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.503203:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908480 00000100:00100000:1.0:1789760792.503218:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908480 00000100:00100000:1.0:1789760792.503303:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722593683185 time 722608147846 state 0x0 ref 1 00000100:00100000:1.0:1789760792.503322:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722608408262 time 722608147846 state 0x1 ref 1 00000100:00100000:1.0:1789760792.503360:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722608408262 time 722627966571 state 0x0 ref 1 00000100:00100000:1.0:1789760792.503378:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.503470:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80605184, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26710 00010000:00010000:1.0:1789760792.503481:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.503513:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.503519:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.503527:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.503544:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.511611:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986986 is committed 00002000:00010000:1.0:1789760792.511767:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80605184 -> 80609280 00002000:00010000:1.0:1789760792.511773:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157456 -> 157464 00010000:00010000:1.0:1789760792.511790:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.511806:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.511817:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.511824:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.511832:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.511841:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.511866:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.511874:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2222 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80605184->80609279] (req 80605184->80609279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.512014:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8630us (9104us total) trans 4294986986 rc 0/0 00000100:00100000:1.0:1789760792.512028:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19690 00000100:00100000:1.0:1789760792.512046:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722608408262 time 722627966571 state 0x2 ref 0 00000100:00100000:2.0:1789760792.523900:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.524006:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908608 00000100:00100000:1.0:1789760792.524020:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908608 00000100:00100000:1.0:1789760792.524109:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722608408262 time 722627966571 state 0x0 ref 1 00000100:00100000:1.0:1789760792.524130:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722628226987 time 722627966571 state 0x1 ref 1 00000100:00100000:1.0:1789760792.524172:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722628226987 time 722648777880 state 0x0 ref 1 00000100:00100000:1.0:1789760792.524188:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503908608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.524270:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80609280, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26718 00010000:00010000:1.0:1789760792.524284:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.524306:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.524311:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.524319:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.524336:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.529825:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986987 is committed 00002000:00010000:1.0:1789760792.530041:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80609280 -> 80613376 00002000:00010000:1.0:1789760792.530048:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157464 -> 157472 00010000:00010000:1.0:1789760792.530065:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.530080:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.530090:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.530098:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.530104:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.530114:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.530140:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.530149:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2229 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80609280->80613375] (req 80609280->80613375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.530377:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503908608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6183us (6482us total) trans 4294986987 rc 0/0 00000100:00100000:1.0:1789760792.530440:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19691 00000100:00100000:1.0:1789760792.530454:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722628226987 time 722648777880 state 0x2 ref 0 00000100:00100000:2.0:1789760792.539590:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.539725:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908736 00000100:00100000:1.0:1789760792.539740:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908736 00000100:00100000:1.0:1789760792.539819:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722628226987 time 722648777880 state 0x0 ref 1 00000100:00100000:1.0:1789760792.539840:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722649038296 time 722648777880 state 0x1 ref 1 00000100:00100000:1.0:1789760792.539882:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722649038296 time 722664488547 state 0x0 ref 1 00000100:00100000:1.0:1789760792.540252:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.540326:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80613376, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26720 00010000:00010000:1.0:1789760792.540339:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.540360:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.540367:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.540374:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.540479:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.546761:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986988 is committed 00002000:00010000:1.0:1789760792.546911:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80613376 -> 80617472 00002000:00010000:1.0:1789760792.546917:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157472 -> 157480 00010000:00010000:1.0:1789760792.546935:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.546951:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.546962:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.546969:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.546975:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.546985:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.547010:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.547025:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2230 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80613376->80617471] (req 80613376->80617471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.547283:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7380us (7700us total) trans 4294986988 rc 0/0 00000100:00100000:1.0:1789760792.547295:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19692 00000100:00100000:1.0:1789760792.547307:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722649038296 time 722664488547 state 0x2 ref 0 00000100:00100000:2.0:1789760792.552882:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.554710:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908864 00000100:00100000:0.0:1789760792.554731:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908864 00000100:00100000:0.0:1789760792.554825:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722649038296 time 722664488547 state 0x0 ref 1 00000100:00100000:0.0:1789760792.554846:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722664748963 time 722664488547 state 0x1 ref 1 00000100:00100000:0.0:1789760792.554893:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722664748963 time 722679499258 state 0x0 ref 1 00000100:00100000:0.0:1789760792.554909:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.554977:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80617472, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26728 00010000:00010000:0.0:1789760792.554990:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.555011:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.555016:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.555025:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.555043:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.560108:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986989 is committed 00002000:00010000:0.0:1789760792.560256:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80617472 -> 80621568 00002000:00010000:0.0:1789760792.560262:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157480 -> 157488 00010000:00010000:0.0:1789760792.560278:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.560298:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.560308:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.560317:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.560323:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.560334:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.560360:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.560368:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2237 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80617472->80621567] (req 80617472->80621567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.560543:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5622us (7661us total) trans 4294986989 rc 0/0 00000100:00100000:0.0:1789760792.560555:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19693 00000100:00100000:0.0:1789760792.560566:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722664748963 time 722679499258 state 0x2 ref 0 00000100:00100000:2.0:1789760792.563944:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.564746:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503908992 00000100:00100000:1.0:1789760792.564761:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503908992 00000100:00100000:1.0:1789760792.564842:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722664748963 time 722679499258 state 0x0 ref 1 00000100:00100000:1.0:1789760792.564863:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722679759674 time 722679499258 state 0x1 ref 1 00000100:00100000:1.0:1789760792.564951:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722679759674 time 722689555773 state 0x0 ref 1 00000100:00100000:1.0:1789760792.564968:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.565047:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80621568, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26730 00010000:00010000:1.0:1789760792.565068:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.565089:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.565095:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.565103:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.565126:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.572763:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986990 is committed 00002000:00010000:1.0:1789760792.573290:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80621568 -> 80625664 00002000:00010000:1.0:1789760792.573297:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157488 -> 157496 00010000:00010000:1.0:1789760792.573315:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.573328:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.573337:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.573346:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.573351:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.573361:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.573385:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.573845:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f223e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80621568->80625663] (req 80621568->80625663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.575003:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503908992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10025us (11061us total) trans 4294986990 rc 0/0 00000100:00100000:0.0:1789760792.575020:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19694 00000100:00100000:0.0:1789760792.575033:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722679759674 time 722689555773 state 0x2 ref 0 00000100:00100000:2.0:1789760792.578970:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.579071:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503909120 00000100:00100000:1.0:1789760792.579084:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503909120 00000100:00100000:1.0:1789760792.579169:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722679759674 time 722689555773 state 0x0 ref 1 00000100:00100000:1.0:1789760792.579187:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722689816189 time 722689555773 state 0x1 ref 1 00000100:00100000:1.0:1789760792.579224:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722689816189 time 722703830629 state 0x0 ref 1 00000100:00100000:1.0:1789760792.579239:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503909120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.579304:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80625664, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26738 00010000:00010000:1.0:1789760792.579316:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.579335:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.579340:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.579347:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.579363:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.585272:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986991 is committed 00002000:00010000:2.0:1789760792.585478:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80625664 -> 80629760 00002000:00010000:2.0:1789760792.585495:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157496 -> 157504 00010000:00010000:2.0:1789760792.585513:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.585813:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.585824:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.585832:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.585838:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.585848:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.585874:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.585888:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2245 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80625664->80629759] (req 80625664->80629759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.586105:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503909120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6861us (7143us total) trans 4294986991 rc 0/0 00000100:00100000:2.0:1789760792.586119:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19695 00000100:00100000:2.0:1789760792.586132:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722689816189 time 722703830629 state 0x2 ref 0 00000100:00100000:2.0:1789760792.597387:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760792.598868:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503909248 00000100:00100000:3.0:1789760792.598887:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503909248 00000100:00100000:3.0:1789760792.598978:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722689816189 time 722703830629 state 0x0 ref 1 00000100:00100000:3.0:1789760792.598998:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722704091045 time 722703830629 state 0x1 ref 1 00000100:00100000:3.0:1789760792.599050:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722704091045 time 722723655824 state 0x0 ref 1 00000100:00100000:3.0:1789760792.599300:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503909248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760792.599365:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80629760, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26740 00010000:00010000:3.0:1789760792.599376:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.599458:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760792.599464:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.599471:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.599499:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.611543:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986992 is committed 00002000:00010000:3.0:1789760792.612330:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80629760 -> 80633856 00002000:00010000:3.0:1789760792.612339:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157504 -> 157512 00010000:00010000:3.0:1789760792.612357:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.612371:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.612405:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.612414:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760792.612420:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.612431:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.612455:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.612464:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f224c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80629760->80633855] (req 80629760->80633855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.612843:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503909248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13543us (15457us total) trans 4294986992 rc 0/0 00000100:00100000:3.0:1789760792.612857:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19696 00000100:00100000:3.0:1789760792.612870:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722704091045 time 722723655824 state 0x2 ref 0 00000100:00100000:1.0:1789760792.619590:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.620292:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503909376 00000100:00100000:0.0:1789760792.620306:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503909376 00000100:00100000:0.0:1789760792.621480:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722704091045 time 722723655824 state 0x0 ref 1 00000100:00100000:0.0:1789760792.621594:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722723916240 time 722723655824 state 0x1 ref 1 00000100:00100000:2.0:1789760792.621796:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722723916240 time 722746397749 state 0x0 ref 1 00000100:00100000:2.0:1789760792.621827:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503909376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.621917:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80633856, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26748 00010000:00010000:2.0:1789760792.621930:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.621952:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.621958:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.621966:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.621982:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.627338:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986993 is committed 00002000:00010000:2.0:1789760792.627814:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80633856 -> 80637952 00002000:00010000:2.0:1789760792.627820:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157512 -> 157520 00010000:00010000:2.0:1789760792.627837:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.627852:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.627862:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.627870:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.627876:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.628343:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.628381:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.628422:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2253 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80633856->80637951] (req 80633856->80637951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.628750:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503909376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6930us (9166us total) trans 4294986993 rc 0/0 00000100:00100000:2.0:1789760792.628765:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19697 00000100:00100000:2.0:1789760792.628778:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722723916240 time 722746397749 state 0x2 ref 0 00000100:00100000:1.0:1789760792.635999:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.636315:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503909504 00000100:00100000:0.0:1789760792.636329:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503909504 00000100:00100000:0.0:1789760792.636667:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722723916240 time 722746397749 state 0x0 ref 1 00000100:00100000:0.0:1789760792.636690:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722746658165 time 722746397749 state 0x1 ref 1 00000100:00100000:3.0:1789760792.636848:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722746658165 time 722761451432 state 0x0 ref 1 00000100:00100000:3.0:1789760792.636867:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503909504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760792.636937:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80637952, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26750 00010000:00010000:3.0:1789760792.636950:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.636970:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760792.636975:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.636983:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.637069:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.654105:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986994 is committed 00002000:00010000:2.0:1789760792.654779:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80637952 -> 80642048 00002000:00010000:2.0:1789760792.654785:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157520 -> 157528 00010000:00010000:2.0:1789760792.654799:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.654813:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.654834:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.654841:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.654845:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.654854:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.654875:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.654882:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f225a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80637952->80642047] (req 80637952->80642047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.655095:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503909504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18228us (19109us total) trans 4294986994 rc 0/0 00000100:00100000:2.0:1789760792.655105:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19698 00000100:00100000:2.0:1789760792.655115:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722746658165 time 722761451432 state 0x2 ref 0 00000100:00100000:0.0:1789760792.660894:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.661038:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503909632 00000100:00100000:1.0:1789760792.661056:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503909632 00000100:00100000:1.0:1789760792.661116:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722746658165 time 722761451432 state 0x0 ref 1 00000100:00100000:1.0:1789760792.661132:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722761711848 time 722761451432 state 0x1 ref 1 00000100:00100000:1.0:1789760792.661208:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722761711848 time 722785812439 state 0x0 ref 1 00000100:00100000:1.0:1789760792.661232:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503909632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.661283:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80642048, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26758 00010000:00010000:1.0:1789760792.661292:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.661309:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.661313:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.661319:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.661331:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.669017:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986995 is committed 00002000:00010000:1.0:1789760792.669508:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80642048 -> 80646144 00002000:00010000:1.0:1789760792.669635:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157528 -> 157536 00010000:00010000:1.0:1789760792.669656:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.669671:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.669686:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.669696:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.669701:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.669712:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.669742:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.669750:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2261 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80642048->80646143] (req 80642048->80646143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.669874:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503909632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8638us (8989us total) trans 4294986995 rc 0/0 00000100:00100000:1.0:1789760792.669885:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19699 00000100:00100000:1.0:1789760792.669897:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722761711848 time 722785812439 state 0x2 ref 0 00000100:00100000:1.0:1789760792.673742:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.674184:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503909760 00000100:00100000:2.0:1789760792.674199:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503909760 00000100:00100000:2.0:1789760792.674261:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722761711848 time 722785812439 state 0x0 ref 1 00000100:00100000:2.0:1789760792.674276:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722786072855 time 722785812439 state 0x1 ref 1 00000100:00100000:2.0:1789760792.674314:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722786072855 time 722798921511 state 0x0 ref 1 00000100:00100000:2.0:1789760792.674328:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503909760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.674383:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80646144, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26760 00010000:00010000:2.0:1789760792.674422:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.674438:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.674443:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.674449:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.674462:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.680449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986996 is committed 00002000:00010000:2.0:1789760792.680573:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80646144 -> 80650240 00002000:00010000:2.0:1789760792.680580:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157536 -> 157544 00010000:00010000:2.0:1789760792.680593:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.680606:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.680615:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.680623:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.680627:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.680636:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.680653:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.680660:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2268 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80646144->80650239] (req 80646144->80650239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.680749:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503909760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6419us (7017us total) trans 4294986996 rc 0/0 00000100:00100000:2.0:1789760792.680760:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19700 00000100:00100000:2.0:1789760792.680771:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722786072855 time 722798921511 state 0x2 ref 0 00000100:00100000:2.0:1789760792.689702:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.689891:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503909888 00000100:00100000:0.0:1789760792.689908:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503909888 00000100:00100000:0.0:1789760792.689987:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722786072855 time 722798921511 state 0x0 ref 1 00000100:00100000:0.0:1789760792.690006:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722799181927 time 722798921511 state 0x1 ref 1 00000100:00100000:0.0:1789760792.690051:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722799181927 time 722814657047 state 0x0 ref 1 00000100:00100000:0.0:1789760792.690065:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503909888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.690133:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80650240, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26768 00010000:00010000:0.0:1789760792.690145:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.690165:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.690171:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.690180:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.690196:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.696066:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986997 is committed 00002000:00010000:0.0:1789760792.696246:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80650240 -> 80654336 00002000:00010000:0.0:1789760792.696253:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157544 -> 157552 00010000:00010000:0.0:1789760792.696270:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.696419:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.696431:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.696438:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.696444:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.696459:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.696497:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.696506:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f226f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80650240->80654335] (req 80650240->80654335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.696622:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503909888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6553us (6921us total) trans 4294986997 rc 0/0 00000100:00100000:0.0:1789760792.696635:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19701 00000100:00100000:0.0:1789760792.696648:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722799181927 time 722814657047 state 0x2 ref 0 00000100:00100000:0.0:1789760792.701690:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.701786:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910016 00000100:00100000:2.0:1789760792.701799:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910016 00000100:00100000:2.0:1789760792.701860:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722799181927 time 722814657047 state 0x0 ref 1 00000100:00100000:2.0:1789760792.701874:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722814917463 time 722814657047 state 0x1 ref 1 00000100:00100000:2.0:1789760792.701897:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722814917463 time 722826503826 state 0x0 ref 1 00000100:00100000:2.0:1789760792.701912:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503910016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.701956:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80654336, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26770 00010000:00010000:2.0:1789760792.701965:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.701980:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.701985:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.701990:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.702004:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.710931:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986998 is committed 00002000:00010000:3.0:1789760792.712271:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80654336 -> 80658432 00002000:00010000:3.0:1789760792.712282:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157552 -> 157560 00010000:00010000:3.0:1789760792.712301:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.712316:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.712326:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.712335:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760792.712340:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.712352:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.712377:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.712385:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2276 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80654336->80658431] (req 80654336->80658431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.713005:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503910016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11089us (11322us total) trans 4294986998 rc 0/0 00000100:00100000:3.0:1789760792.713019:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19702 00000100:00100000:3.0:1789760792.713032:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722814917463 time 722826503826 state 0x2 ref 0 00000100:00100000:1.0:1789760792.717609:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760792.717712:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910144 00000100:00100000:3.0:1789760792.717725:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910144 00000100:00100000:3.0:1789760792.717810:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722814917463 time 722826503826 state 0x0 ref 1 00000100:00100000:3.0:1789760792.717831:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722826764242 time 722826503826 state 0x1 ref 1 00000100:00100000:3.0:1789760792.717870:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722826764242 time 722842476136 state 0x0 ref 1 00000100:00100000:3.0:1789760792.717885:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503910144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760792.717953:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80658432, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26778 00010000:00010000:3.0:1789760792.717964:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.717986:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760792.717991:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.717998:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.718013:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.726443:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294986999 is committed 00002000:00010000:1.0:1789760792.726614:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80658432 -> 80662528 00002000:00010000:1.0:1789760792.726621:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157560 -> 157568 00010000:00010000:1.0:1789760792.726640:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.726656:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.726666:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.726676:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.726681:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.726693:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.726718:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.726726:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f227d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80658432->80662527] (req 80658432->80662527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.726912:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503910144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9021us (9310us total) trans 4294986999 rc 0/0 00000100:00100000:1.0:1789760792.726928:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19703 00000100:00100000:1.0:1789760792.726940:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722826764242 time 722842476136 state 0x2 ref 0 00000100:00100000:2.0:1789760792.737322:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.737446:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910272 00000100:00100000:1.0:1789760792.737459:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910272 00000100:00100000:1.0:1789760792.737537:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722826764242 time 722842476136 state 0x0 ref 1 00000100:00100000:1.0:1789760792.737554:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722842736552 time 722842476136 state 0x1 ref 1 00000100:00100000:1.0:1789760792.737586:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722842736552 time 722862192661 state 0x0 ref 1 00000100:00100000:1.0:1789760792.737599:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503910272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.737662:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80662528, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26780 00010000:00010000:1.0:1789760792.737673:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.737703:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.737708:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.737715:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.737732:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.744053:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987000 is committed 00002000:00010000:1.0:1789760792.744858:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80662528 -> 80666624 00002000:00010000:1.0:1789760792.744866:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157568 -> 157576 00010000:00010000:1.0:1789760792.744885:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.744901:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.744911:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.744919:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.744924:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.744934:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.744960:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.744967:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2284 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80662528->80666623] (req 80662528->80666623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.745442:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503910272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7789us (8078us total) trans 4294987000 rc 0/0 00000100:00100000:1.0:1789760792.745456:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19704 00000100:00100000:1.0:1789760792.745468:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722842736552 time 722862192661 state 0x2 ref 0 00000100:00100000:0.0:1789760792.749686:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.750341:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910400 00000100:00100000:2.0:1789760792.750357:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910400 00000100:00100000:2.0:1789760792.750497:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722842736552 time 722862192661 state 0x0 ref 1 00000100:00100000:2.0:1789760792.750519:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722862453077 time 722862192661 state 0x1 ref 1 00000100:00100000:2.0:1789760792.750769:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722862453077 time 722875372792 state 0x0 ref 1 00000100:00100000:2.0:1789760792.750783:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503910400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.750854:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80666624, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26788 00010000:00010000:2.0:1789760792.750867:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.750892:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760792.750898:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.750906:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.750924:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.759985:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987001 is committed 00002000:00010000:0.0:1789760792.760138:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80666624 -> 80670720 00002000:00010000:0.0:1789760792.760144:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157576 -> 157584 00010000:00010000:0.0:1789760792.760162:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.760177:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.760187:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.760197:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.760205:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.760228:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.760252:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.760261:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f228b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80666624->80670719] (req 80666624->80670719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.760454:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503910400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9604us (10722us total) trans 4294987001 rc 0/0 00000100:00100000:0.0:1789760792.760467:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19705 00000100:00100000:0.0:1789760792.762456:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722862453077 time 722875372792 state 0x2 ref 0 00000100:00100000:2.0:1789760792.767055:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760792.767703:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910528 00000100:00100000:3.0:1789760792.767720:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910528 00000100:00100000:3.0:1789760792.767796:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722862453077 time 722875372792 state 0x0 ref 1 00000100:00100000:3.0:1789760792.767815:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722875633208 time 722875372792 state 0x1 ref 1 00000100:00100000:3.0:1789760792.767838:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722875633208 time 722892445125 state 0x0 ref 1 00000100:00100000:3.0:1789760792.767852:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760792.767916:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80670720, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26790 00010000:00010000:3.0:1789760792.767928:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.767951:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760792.767957:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.767965:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.767983:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.780835:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987002 is committed 00002000:00010000:2.0:1789760792.780997:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80670720 -> 80674816 00002000:00010000:2.0:1789760792.781003:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157584 -> 157592 00010000:00010000:2.0:1789760792.781020:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.781041:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.781050:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.781059:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.781064:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.781074:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.781099:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.781107:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2292 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80670720->80674815] (req 80670720->80674815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.781248:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13390us (14202us total) trans 4294987002 rc 0/0 00000100:00100000:2.0:1789760792.781262:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19706 00000100:00100000:2.0:1789760792.781274:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722875633208 time 722892445125 state 0x2 ref 0 00000100:00100000:2.0:1789760792.798328:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.798509:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910656 00000100:00100000:1.0:1789760792.798524:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910656 00000100:00100000:1.0:1789760792.798602:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722875633208 time 722892445125 state 0x0 ref 1 00000100:00100000:1.0:1789760792.798621:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722892705541 time 722892445125 state 0x1 ref 1 00000100:00100000:1.0:1789760792.798671:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722892705541 time 722923276260 state 0x0 ref 1 00000100:00100000:1.0:1789760792.798687:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.798757:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80674816, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x26798 00010000:00010000:1.0:1789760792.798768:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.798789:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.798794:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.798802:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.798819:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.810657:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987003 is committed 00002000:00010000:0.0:1789760792.810823:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80674816 -> 80678912 00002000:00010000:0.0:1789760792.810829:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:0.0:1789760792.810832:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00002000:00010000:0.0:1789760792.810834:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157592 -> 157600 00010000:00010000:0.0:1789760792.810852:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.810874:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.810884:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.810892:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.810897:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.810906:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.810929:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.810936:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2299 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80674816->80678911] (req 80674816->80678911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.811062:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12371us (12743us total) trans 4294987003 rc 0/0 00000100:00100000:0.0:1789760792.811074:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19707 00000100:00100000:0.0:1789760792.811087:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722892705541 time 722923276260 state 0x2 ref 0 00000100:00100000:0.0:1789760792.825817:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.825936:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910784 00000100:00100000:1.0:1789760792.825949:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910784 00000100:00100000:1.0:1789760792.826026:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722892705541 time 722923276260 state 0x0 ref 1 00000100:00100000:1.0:1789760792.826045:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722923536676 time 722923276260 state 0x1 ref 1 00000100:00100000:1.0:1789760792.826083:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722923536676 time 722950689405 state 0x0 ref 1 00000100:00100000:1.0:1789760792.826096:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.826164:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80678912, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267a0 00010000:00010000:1.0:1789760792.826177:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.826197:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760792.826203:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.826222:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.826239:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.833006:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987004 is committed 00002000:00010000:1.0:1789760792.833386:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80678912 -> 80683008 00002000:00010000:1.0:1789760792.833416:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157600 -> 157608 00010000:00010000:1.0:1789760792.833434:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.833449:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.833458:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.833471:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760792.833656:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.833668:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.833762:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.833772:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80678912->80683007] (req 80678912->80683007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.833963:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7860us (8154us total) trans 4294987004 rc 0/0 00000100:00100000:1.0:1789760792.833976:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19708 00000100:00100000:1.0:1789760792.833988:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722923536676 time 722950689405 state 0x2 ref 0 00000100:00100000:2.0:1789760792.839608:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.839946:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503910912 00000100:00100000:0.0:1789760792.839958:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503910912 00000100:00100000:0.0:1789760792.840026:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722923536676 time 722950689405 state 0x0 ref 1 00000100:00100000:0.0:1789760792.840041:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722950949821 time 722950689405 state 0x1 ref 1 00000100:00100000:0.0:1789760792.840073:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722950949821 time 722964678535 state 0x0 ref 1 00000100:00100000:0.0:1789760792.840086:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.840159:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80683008, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267a8 00010000:00010000:0.0:1789760792.840170:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.840191:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.840197:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.840205:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.840292:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.846753:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987005 is committed 00002000:00010000:2.0:1789760792.846910:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80683008 -> 80687104 00002000:00010000:2.0:1789760792.846917:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157608 -> 157616 00010000:00010000:2.0:1789760792.846934:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.846949:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.846960:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.846968:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.846974:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.846983:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.847007:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.847014:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f22a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80683008->80687103] (req 80683008->80687103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.847179:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503910912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7087us (7581us total) trans 4294987005 rc 0/0 00000100:00100000:2.0:1789760792.847196:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19709 00000100:00100000:2.0:1789760792.847273:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722950949821 time 722964678535 state 0x2 ref 0 00000100:00100000:2.0:1789760792.852814:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.852899:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911040 00000100:00100000:0.0:1789760792.852913:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911040 00000100:00100000:0.0:1789760792.852990:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722950949821 time 722964678535 state 0x0 ref 1 00000100:00100000:0.0:1789760792.853010:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722964938951 time 722964678535 state 0x1 ref 1 00000100:00100000:0.0:1789760792.853105:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722964938951 time 722977710285 state 0x0 ref 1 00000100:00100000:0.0:1789760792.853120:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.853184:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80687104, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267b0 00010000:00010000:0.0:1789760792.853196:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.853229:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760792.853235:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.853242:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.853258:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.861295:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987006 is committed 00002000:00010000:0.0:1789760792.861437:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80687104 -> 80691200 00002000:00010000:0.0:1789760792.861442:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157616 -> 157624 00010000:00010000:0.0:1789760792.861457:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.861468:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.861492:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.861508:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.861514:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.861524:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.861550:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.861559:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f22ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80687104->80691199] (req 80687104->80691199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.861807:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8682us (9000us total) trans 4294987006 rc 0/0 00000100:00100000:0.0:1789760792.861821:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19710 00000100:00100000:0.0:1789760792.861833:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722964938951 time 722977710285 state 0x2 ref 0 00000100:00100000:0.0:1789760792.870216:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760792.870650:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911168 00000100:00100000:3.0:1789760792.870666:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911168 00000100:00100000:3.0:1789760792.870745:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722964938951 time 722977710285 state 0x0 ref 1 00000100:00100000:3.0:1789760792.870766:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722977970701 time 722977710285 state 0x1 ref 1 00000100:00100000:3.0:1789760792.870807:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722977970701 time 722995412747 state 0x0 ref 1 00000100:00100000:3.0:1789760792.870823:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503911168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760792.870893:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80691200, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267b8 00010000:00010000:3.0:1789760792.870905:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.870924:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760792.870930:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.870938:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.870956:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.888246:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987007 is committed 00002000:00010000:0.0:1789760792.888433:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80691200 -> 80695296 00002000:00010000:0.0:1789760792.888441:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157624 -> 157632 00010000:00010000:0.0:1789760792.888458:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.888473:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.888495:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.888503:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.888509:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.888519:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.888548:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.888556:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f22b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80691200->80695295] (req 80691200->80695295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.891589:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503911168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 20749us (21377us total) trans 4294987007 rc 0/0 00000100:00100000:0.0:1789760792.891608:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19711 00000100:00100000:0.0:1789760792.892032:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722977970701 time 722995412747 state 0x2 ref 0 00000100:00100000:2.0:1789760792.898892:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760792.899037:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911296 00000100:00100000:1.0:1789760792.899048:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911296 00000100:00100000:1.0:1789760792.899110:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722977970701 time 722995412747 state 0x0 ref 1 00000100:00100000:1.0:1789760792.899124:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722995673163 time 722995412747 state 0x1 ref 1 00000100:00100000:1.0:1789760792.899153:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722995673163 time 723023759247 state 0x0 ref 1 00000100:00100000:1.0:1789760792.899169:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503911296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760792.899238:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80695296, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267c0 00010000:00010000:1.0:1789760792.899261:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.899282:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760792.899344:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.899352:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.899373:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.908581:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987008 is committed 00002000:00010000:2.0:1789760792.908946:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80695296 -> 80699392 00002000:00010000:2.0:1789760792.908954:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157632 -> 157640 00010000:00010000:2.0:1789760792.908970:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.908984:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.908994:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.909003:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.909008:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.909018:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.909056:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.909070:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f22bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80695296->80699391] (req 80695296->80699391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.909202:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503911296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10027us (10317us total) trans 4294987008 rc 0/0 00000100:00100000:2.0:1789760792.909215:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19712 00000100:00100000:2.0:1789760792.909228:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722995673163 time 723023759247 state 0x2 ref 0 00000100:00100000:2.0:1789760792.916760:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760792.916868:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911424 00000100:00100000:0.0:1789760792.916881:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911424 00000100:00100000:0.0:1789760792.916960:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 722995673163 time 723023759247 state 0x0 ref 1 00000100:00100000:0.0:1789760792.916979:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723024019663 time 723023759247 state 0x1 ref 1 00000100:00100000:0.0:1789760792.917069:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723024019663 time 723041674616 state 0x0 ref 1 00000100:00100000:0.0:1789760792.917084:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760792.917150:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80699392, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267c8 00010000:00010000:0.0:1789760792.917162:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.917182:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760792.917194:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.917202:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.917223:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.924382:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987009 is committed 00002000:00010000:0.0:1789760792.924676:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80699392 -> 80703488 00002000:00010000:0.0:1789760792.924683:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157640 -> 157648 00010000:00010000:0.0:1789760792.924703:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.924717:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.924727:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.924742:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760792.924749:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.924759:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.924785:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.924794:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f22c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80699392->80703487] (req 80699392->80703487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.924911:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7824us (8162us total) trans 4294987009 rc 0/0 00000100:00100000:0.0:1789760792.924924:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19713 00000100:00100000:0.0:1789760792.924938:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723024019663 time 723041674616 state 0x2 ref 0 00000100:00100000:0.0:1789760792.930190:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760792.930374:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911552 00000100:00100000:2.0:1789760792.930503:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911552 00000100:00100000:2.0:1789760792.931757:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723024019663 time 723041674616 state 0x0 ref 1 00000100:00100000:2.0:1789760792.931780:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723041935032 time 723041674616 state 0x1 ref 1 00000100:00100000:2.0:1789760792.931867:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723041935032 time 723056472508 state 0x0 ref 1 00000100:00100000:2.0:1789760792.931887:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760792.931951:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80703488, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267d0 00010000:00010000:2.0:1789760792.931962:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.931984:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760792.931989:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.931995:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.932013:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.948366:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987010 is committed 00002000:00010000:2.0:1789760792.949694:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80703488 -> 80707584 00002000:00010000:2.0:1789760792.949706:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157648 -> 157656 00010000:00010000:2.0:1789760792.949727:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.949743:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.949754:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.949763:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.949769:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.949780:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.949804:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.949812:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80703488->80707583] (req 80703488->80707583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.950347:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18456us (20157us total) trans 4294987010 rc 0/0 00000100:00100000:2.0:1789760792.950363:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19714 00000100:00100000:2.0:1789760792.950376:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723041935032 time 723056472508 state 0x2 ref 0 00000100:00100000:2.0:1789760792.957092:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760792.957175:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911680 00000100:00100000:3.0:1789760792.957190:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911680 00000100:00100000:3.0:1789760792.957269:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723041935032 time 723056472508 state 0x0 ref 1 00000100:00100000:3.0:1789760792.957289:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723056732924 time 723056472508 state 0x1 ref 1 00000100:00100000:3.0:1789760792.957326:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723056732924 time 723081931991 state 0x0 ref 1 00000100:00100000:3.0:1789760792.957339:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760792.957441:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80707584, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267d8 00010000:00010000:3.0:1789760792.957454:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.957474:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760792.957480:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.957499:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.957515:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.968100:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987011 is committed 00002000:00010000:3.0:1789760792.968350:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80707584 -> 80711680 00002000:00010000:3.0:1789760792.968357:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157656 -> 157664 00010000:00010000:3.0:1789760792.968374:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.968861:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.968878:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.968886:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760792.968893:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.968905:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.968933:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.968942:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f22d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80707584->80711679] (req 80707584->80711679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.969775:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503911680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12423us (12684us total) trans 4294987011 rc 0/0 00000100:00100000:2.0:1789760792.969792:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19715 00000100:00100000:2.0:1789760792.969814:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723056732924 time 723081931991 state 0x2 ref 0 00000100:00100000:2.0:1789760792.975530:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760792.975729:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911808 00000100:00100000:3.0:1789760792.975743:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911808 00000100:00100000:3.0:1789760792.975828:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723056732924 time 723081931991 state 0x0 ref 1 00000100:00100000:3.0:1789760792.975848:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723082192407 time 723081931991 state 0x1 ref 1 00000100:00100000:3.0:1789760792.975886:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723082192407 time 723100491918 state 0x0 ref 1 00000100:00100000:3.0:1789760792.975902:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503911808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760792.975972:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80711680, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267e0 00010000:00010000:3.0:1789760792.975983:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.976005:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760792.976011:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.976020:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.976037:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.989432:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987012 is committed 00002000:00010000:2.0:1789760792.989582:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80711680 -> 80715776 00002000:00010000:2.0:1789760792.989589:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157664 -> 157672 00010000:00010000:2.0:1789760792.989605:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.989621:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.989744:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.989754:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760792.989768:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.989780:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.989804:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.989813:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f22d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80711680->80715775] (req 80711680->80715775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.989934:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503911808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14028us (14420us total) trans 4294987012 rc 0/0 00000100:00100000:2.0:1789760792.989946:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19716 00000100:00100000:2.0:1789760792.989961:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723082192407 time 723100491918 state 0x2 ref 0 00000100:00100000:0.0:1789760793.001498:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.001959:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503911936 00000100:00100000:2.0:1789760793.001973:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503911936 00000100:00100000:2.0:1789760793.002051:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723082192407 time 723100491918 state 0x0 ref 1 00000100:00100000:2.0:1789760793.002071:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723100752334 time 723100491918 state 0x1 ref 1 00000100:00100000:2.0:1789760793.002107:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723100752334 time 723126712962 state 0x0 ref 1 00000100:00100000:2.0:1789760793.002121:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503911936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.002189:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80715776, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267e8 00010000:00010000:2.0:1789760793.002200:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.002220:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.002225:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.002233:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.002250:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.011979:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987013 is committed 00002000:00010000:2.0:1789760793.012324:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80715776 -> 80719872 00002000:00010000:2.0:1789760793.012331:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157672 -> 157680 00010000:00010000:2.0:1789760793.012349:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.012365:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.012374:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.012383:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.012436:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.012448:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.012480:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.012503:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80715776->80719871] (req 80715776->80719871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.012634:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503911936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10508us (11152us total) trans 4294987013 rc 0/0 00000100:00100000:2.0:1789760793.012645:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19717 00000100:00100000:2.0:1789760793.012658:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723100752334 time 723126712962 state 0x2 ref 0 00000100:00100000:1.0:1789760793.018010:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.018245:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912064 00000100:00100000:2.0:1789760793.018259:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912064 00000100:00100000:2.0:1789760793.018339:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723100752334 time 723126712962 state 0x0 ref 1 00000100:00100000:2.0:1789760793.018358:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723126973378 time 723126712962 state 0x1 ref 1 00000100:00100000:2.0:1789760793.018431:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723126973378 time 723143036536 state 0x0 ref 1 00000100:00100000:2.0:1789760793.018446:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503912064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.018529:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80719872, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267f0 00010000:00010000:2.0:1789760793.018540:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.018560:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.018566:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.018574:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.018591:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.025575:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987014 is committed 00002000:00010000:2.0:1789760793.025851:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80719872 -> 80723968 00002000:00010000:2.0:1789760793.025858:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157680 -> 157688 00010000:00010000:2.0:1789760793.025875:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.025891:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.025901:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.025909:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.025986:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.025998:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.026024:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.026037:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f22e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80719872->80723967] (req 80719872->80723967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.026468:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503912064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8018us (8468us total) trans 4294987014 rc 0/0 00000100:00100000:2.0:1789760793.026480:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19718 00000100:00100000:2.0:1789760793.026505:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723126973378 time 723143036536 state 0x2 ref 0 00000100:00100000:1.0:1789760793.031434:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.031518:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.031728:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.031759:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.031807:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912192 00000100:00100000:3.0:1789760793.031820:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912192 00000100:00100000:3.0:1789760793.031895:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 494 00000100:00100000:3.0:1789760793.031906:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503912192:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760793.032018:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503912192:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 113us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.032026:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 494 00000100:00100000:0.0:1789760793.034257:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912320 00000100:00100000:2.0:1789760793.034731:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912576 00000100:00100000:2.0:1789760793.034737:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912576 00000100:00100000:2.0:1789760793.034828:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912448 00000100:00100000:2.0:1789760793.034832:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912448 00000100:00080000:3.0:1789760793.034845:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760793.034850:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760793.034864:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 991 00000100:00100000:2.0:1789760793.034876:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503912576:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760793.034896:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912320 00000100:00100000:2.0:1789760793.034977:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503912576:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 103us (3457us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.035000:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 992 00000100:00100000:3.0:1789760793.035011:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503912448:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:3.0:1789760793.035060:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503912448:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 51us (3300us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.035067:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 992 00000100:00100000:3.0:1789760793.035128:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.035129:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.035146:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912704 00000100:00100000:3.0:1789760793.035150:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912704 00000100:00100000:2.0:1789760793.035159:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 991 00000100:00100000:3.0:1789760793.035171:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 993 00000100:00100000:3.0:1789760793.035180:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503912704:12345-192.168.203.58@tcp:400:kworker.0 00000100:00080000:1.0:1789760793.035218:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760793.035218:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503912704:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 30us (80us total) trans 0 rc 0/0 00000100:00080000:1.0:1789760793.035221:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760793.035225:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 993 00000100:00100000:1.0:1789760793.035237:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912832 00000100:00100000:1.0:1789760793.035241:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912832 00000100:00100000:1.0:1789760793.035290:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723126973378 time 723143036536 state 0x0 ref 1 00000100:00100000:1.0:1789760793.035307:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723143296952 time 723143036536 state 0x1 ref 1 00000100:00100000:1.0:1789760793.035327:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723143296952 time 723159933768 state 0x0 ref 1 00000100:00100000:1.0:1789760793.035338:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503912832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.035545:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80723968, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x267f8 00010000:00010000:1.0:1789760793.035559:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.035579:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.035584:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.035591:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.035609:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.036456:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 994 00000100:00100000:0.0:1789760793.036473:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503912320:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760793.036567:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503912320:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 99us (5148us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760793.036575:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 994 00000001:00080000:3.0:1789760793.046051:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987015 is committed 00002000:00010000:1.0:1789760793.046442:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80723968 -> 80728064 00002000:00010000:1.0:1789760793.046448:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157688 -> 157696 00010000:00010000:1.0:1789760793.046467:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046495:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046506:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046514:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.046528:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046540:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046564:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.046572:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f22ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80723968->80728063] (req 80723968->80728063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.046715:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503912832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11370us (11579us total) trans 4294987015 rc 0/0 00000100:00100000:1.0:1789760793.046727:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19719 00000100:00100000:1.0:1789760793.046743:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723143296952 time 723159933768 state 0x2 ref 0 00000100:00100000:3.0:1789760793.051691:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.051828:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503912960 00000100:00100000:2.0:1789760793.051841:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503912960 00000100:00100000:2.0:1789760793.052000:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723143296952 time 723159933768 state 0x0 ref 1 00000100:00100000:2.0:1789760793.052024:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723160194184 time 723159933768 state 0x1 ref 1 00000100:00100000:2.0:1789760793.052063:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723160194184 time 723176668853 state 0x0 ref 1 00000100:00100000:2.0:1789760793.052078:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503912960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.052153:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80728064, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26800 00010000:00010000:2.0:1789760793.052165:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.052185:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.052191:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.052199:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.052216:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.058259:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987016 is committed 00002000:00010000:2.0:1789760793.058697:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80728064 -> 80732160 00002000:00010000:2.0:1789760793.058704:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157696 -> 157704 00010000:00010000:2.0:1789760793.058723:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.058738:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.058748:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.058756:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.058761:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.058775:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.058799:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.058807:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f22f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80728064->80732159] (req 80728064->80732159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.058940:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503912960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6857us (7260us total) trans 4294987016 rc 0/0 00000100:00100000:2.0:1789760793.058953:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19720 00000100:00100000:2.0:1789760793.058966:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723160194184 time 723176668853 state 0x2 ref 0 00000100:00100000:0.0:1789760793.064181:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.064350:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913088 00000100:00100000:3.0:1789760793.064363:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913088 00000100:00100000:3.0:1789760793.064475:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723160194184 time 723176668853 state 0x0 ref 1 00000100:00100000:3.0:1789760793.064502:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723176929269 time 723176668853 state 0x1 ref 1 00000100:00100000:1.0:1789760793.064666:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723176929269 time 723189268438 state 0x0 ref 1 00000100:00100000:1.0:1789760793.064684:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503913088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.064751:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80732160, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26808 00010000:00010000:1.0:1789760793.064763:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.064783:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.064788:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.064796:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.064813:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.075361:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987017 is committed 00002000:00010000:1.0:1789760793.075929:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80732160 -> 80736256 00002000:00010000:1.0:1789760793.075937:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157704 -> 157712 00010000:00010000:1.0:1789760793.075955:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.075971:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.075981:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.075990:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.075996:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.076012:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.076043:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.076050:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f22fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80732160->80736255] (req 80732160->80736255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.076442:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503913088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11710us (12347us total) trans 4294987017 rc 0/0 00000100:00100000:1.0:1789760793.076456:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19721 00000100:00100000:1.0:1789760793.076491:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723176929269 time 723189268438 state 0x2 ref 0 00000100:00100000:1.0:1789760793.081203:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.081303:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913216 00000100:00100000:0.0:1789760793.081326:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913216 00000100:00100000:0.0:1789760793.081435:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723176929269 time 723189268438 state 0x0 ref 1 00000100:00100000:0.0:1789760793.081453:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723189528854 time 723189268438 state 0x1 ref 1 00000100:00100000:0.0:1789760793.081602:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723189528854 time 723206206420 state 0x0 ref 1 00000100:00100000:0.0:1789760793.081619:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503913216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.081684:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80736256, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26810 00010000:00010000:0.0:1789760793.081696:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.081717:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.081722:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.081730:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.081747:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.087840:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987018 is committed 00002000:00010000:0.0:1789760793.090741:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80736256 -> 80740352 00002000:00010000:0.0:1789760793.090754:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157712 -> 157720 00010000:00010000:0.0:1789760793.090772:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.090787:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.090797:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.090806:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.090813:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.090823:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.090850:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.090859:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2302 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80736256->80740351] (req 80736256->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.091011:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503913216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9388us (9816us total) trans 4294987018 rc 0/0 00000100:00100000:0.0:1789760793.091025:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19722 00000100:00100000:0.0:1789760793.091090:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723189528854 time 723206206420 state 0x2 ref 0 00000100:00100000:3.0:1789760793.100316:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.100625:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913344 00000100:00100000:2.0:1789760793.100640:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913344 00000100:00100000:2.0:1789760793.100717:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723189528854 time 723206206420 state 0x0 ref 1 00000100:00100000:2.0:1789760793.100736:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723206466836 time 723206206420 state 0x1 ref 1 00000100:00100000:2.0:1789760793.101056:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723206466836 time 723225659901 state 0x0 ref 1 00000100:00100000:2.0:1789760793.101074:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.101156:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80740352, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26818 00010000:00010000:2.0:1789760793.101172:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101192:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.101198:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.101214:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101230:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.106650:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760793.108505:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987019 is committed 00002000:00010000:3.0:1789760793.108808:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80740352 -> 80744448 00002000:00010000:3.0:1789760793.108814:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157720 -> 157728 00010000:00010000:3.0:1789760793.108832:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.108852:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.108861:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.108869:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.108873:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.108883:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.108906:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.108924:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2309 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80740352->80744447] (req 80740352->80744447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.109104:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8019us (9055us total) trans 4294987019 rc 0/0 00000100:00100000:3.0:1789760793.109118:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19723 00000100:00100000:3.0:1789760793.109130:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723206466836 time 723225659901 state 0x2 ref 0 00000100:00100000:1.0:1789760793.118102:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.118449:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913472 00000100:00100000:2.0:1789760793.118634:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913472 00000100:00100000:2.0:1789760793.118716:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723206466836 time 723225659901 state 0x0 ref 1 00000100:00100000:2.0:1789760793.118735:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723225920317 time 723225659901 state 0x1 ref 1 00000100:00100000:2.0:1789760793.118808:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723225920317 time 723243414350 state 0x0 ref 1 00000100:00100000:2.0:1789760793.118823:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503913472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.118883:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80744448, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26820 00010000:00010000:2.0:1789760793.118893:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.118918:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.118923:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.118930:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.118947:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.124833:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987020 is committed 00002000:00010000:3.0:1789760793.125142:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80744448 -> 80748544 00002000:00010000:3.0:1789760793.125149:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157728 -> 157736 00010000:00010000:3.0:1789760793.125166:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.125182:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.125194:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.125201:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.125206:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.125218:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.125242:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.125252:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2310 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80744448->80748543] (req 80744448->80748543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.125437:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503913472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6567us (7301us total) trans 4294987020 rc 0/0 00000100:00100000:3.0:1789760793.125450:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19724 00000100:00100000:3.0:1789760793.125473:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723225920317 time 723243414350 state 0x2 ref 0 00000100:00100000:1.0:1789760793.131253:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.133385:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913600 00000100:00100000:0.0:1789760793.133441:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913600 00000100:00100000:0.0:1789760793.133549:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723225920317 time 723243414350 state 0x0 ref 1 00000100:00100000:0.0:1789760793.133569:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723243674766 time 723243414350 state 0x1 ref 1 00000100:00100000:0.0:1789760793.133615:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723243674766 time 723258212199 state 0x0 ref 1 00000100:00100000:0.0:1789760793.133631:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.133700:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80748544, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26828 00010000:00010000:0.0:1789760793.133712:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.133733:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.133738:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.133746:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.133764:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.146490:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987021 is committed 00002000:00010000:0.0:1789760793.146672:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80748544 -> 80752640 00002000:00010000:0.0:1789760793.146678:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157736 -> 157744 00010000:00010000:0.0:1789760793.146695:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.146717:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.146728:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.146736:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.146741:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.146752:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.146777:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.146786:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2317 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80748544->80752639] (req 80748544->80752639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.146970:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13335us (15724us total) trans 4294987021 rc 0/0 00000100:00100000:0.0:1789760793.146983:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19725 00000100:00100000:0.0:1789760793.146995:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723243674766 time 723258212199 state 0x2 ref 0 00000100:00100000:0.0:1789760793.151185:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.151442:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913728 00000100:00100000:1.0:1789760793.151456:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913728 00000100:00100000:1.0:1789760793.151547:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723243674766 time 723258212199 state 0x0 ref 1 00000100:00100000:1.0:1789760793.151567:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723258472615 time 723258212199 state 0x1 ref 1 00000100:00100000:1.0:1789760793.151600:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723258472615 time 723276206144 state 0x0 ref 1 00000100:00100000:1.0:1789760793.151618:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.151681:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80752640, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26830 00010000:00010000:1.0:1789760793.151693:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.151713:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.151719:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.151728:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.151746:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.158049:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987022 is committed 00002000:00010000:1.0:1789760793.158198:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80752640 -> 80756736 00002000:00010000:1.0:1789760793.158212:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157744 -> 157752 00010000:00010000:1.0:1789760793.158231:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.158246:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.158256:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.158265:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.158272:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.158282:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.158307:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.158315:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f231e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80752640->80756735] (req 80752640->80756735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.158494:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6864us (7309us total) trans 4294987022 rc 0/0 00000100:00100000:1.0:1789760793.158507:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19726 00000100:00100000:1.0:1789760793.158530:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723258472615 time 723276206144 state 0x2 ref 0 00000100:00100000:2.0:1789760793.164632:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.164845:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913856 00000100:00100000:3.0:1789760793.164858:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913856 00000100:00100000:1.0:1789760793.164999:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723258472615 time 723276206144 state 0x0 ref 1 00000100:00100000:1.0:1789760793.165018:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723276466560 time 723276206144 state 0x1 ref 1 00000100:00100000:1.0:1789760793.165092:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723276466560 time 723289697513 state 0x0 ref 1 00000100:00100000:1.0:1789760793.165108:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.165177:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80756736, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26838 00010000:00010000:1.0:1789760793.165188:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165219:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.165225:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.165233:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165250:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.172628:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987023 is committed 00002000:00010000:1.0:1789760793.172776:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80756736 -> 80760832 00002000:00010000:1.0:1789760793.172782:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157752 -> 157760 00010000:00010000:1.0:1789760793.172799:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.172814:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.172824:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.172832:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.172837:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.172847:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.172876:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.172885:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2325 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80756736->80760831] (req 80756736->80760831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.173053:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7941us (8432us total) trans 4294987023 rc 0/0 00000100:00100000:1.0:1789760793.173066:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19727 00000100:00100000:1.0:1789760793.173078:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723276466560 time 723289697513 state 0x2 ref 0 00000100:00100000:2.0:1789760793.177137:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.177977:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503913984 00000100:00100000:3.0:1789760793.177994:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503913984 00000100:00100000:2.0:1789760793.178170:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723276466560 time 723289697513 state 0x0 ref 1 00000100:00100000:2.0:1789760793.178189:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723289957929 time 723289697513 state 0x1 ref 1 00000100:00100000:2.0:1789760793.178311:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723289957929 time 723302915177 state 0x0 ref 1 00000100:00100000:2.0:1789760793.178337:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.178421:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80760832, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26840 00010000:00010000:2.0:1789760793.178434:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.178453:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.178459:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.178467:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.178492:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.184885:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987024 is committed 00002000:00010000:2.0:1789760793.185036:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80760832 -> 80764928 00002000:00010000:2.0:1789760793.185043:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157760 -> 157768 00010000:00010000:2.0:1789760793.185061:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.185076:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.185086:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.185162:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.185168:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.185178:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.185211:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.185225:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f232c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80760832->80764927] (req 80760832->80764927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.185455:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503913984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7114us (8325us total) trans 4294987024 rc 0/0 00000100:00100000:2.0:1789760793.185468:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19728 00000100:00100000:2.0:1789760793.185480:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723289957929 time 723302915177 state 0x2 ref 0 00000100:00100000:2.0:1789760793.189827:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.189942:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503914112 00000100:00100000:0.0:1789760793.189956:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503914112 00000100:00100000:0.0:1789760793.190034:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723289957929 time 723302915177 state 0x0 ref 1 00000100:00100000:0.0:1789760793.190053:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723303175593 time 723302915177 state 0x1 ref 1 00000100:00100000:0.0:1789760793.190130:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723303175593 time 723314735422 state 0x0 ref 1 00000100:00100000:0.0:1789760793.190147:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503914112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.190217:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80764928, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26848 00010000:00010000:0.0:1789760793.190229:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.190248:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.190253:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.190261:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.190279:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.196918:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987025 is committed 00002000:00010000:0.0:1789760793.197059:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80764928 -> 80769024 00002000:00010000:0.0:1789760793.197065:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157768 -> 157776 00010000:00010000:0.0:1789760793.197082:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.197103:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.197112:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.197121:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.197126:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.197136:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.197160:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.197169:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2333 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80764928->80769023] (req 80764928->80769023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.197350:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503914112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7198us (7531us total) trans 4294987025 rc 0/0 00000100:00100000:0.0:1789760793.197363:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19729 00000100:00100000:0.0:1789760793.197376:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723303175593 time 723314735422 state 0x2 ref 0 00000100:00100000:2.0:1789760793.201654:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.201758:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503914240 00000100:00100000:1.0:1789760793.201770:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503914240 00000100:00100000:1.0:1789760793.201843:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723303175593 time 723314735422 state 0x0 ref 1 00000100:00100000:1.0:1789760793.201861:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723314995838 time 723314735422 state 0x1 ref 1 00000100:00100000:1.0:1789760793.201931:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723314995838 time 723326536468 state 0x0 ref 1 00000100:00100000:1.0:1789760793.201946:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503914240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.202016:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80769024, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26850 00010000:00010000:1.0:1789760793.202028:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.202049:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.202056:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.202064:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.202081:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.207336:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987026 is committed 00002000:00010000:1.0:1789760793.207706:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80769024 -> 80773120 00002000:00010000:1.0:1789760793.207713:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157776 -> 157784 00010000:00010000:1.0:1789760793.207734:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.207747:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.207757:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.207765:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.207771:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.207781:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.207808:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.207822:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f233a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80769024->80773119] (req 80769024->80773119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.207990:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503914240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6038us (6343us total) trans 4294987026 rc 0/0 00000100:00100000:1.0:1789760793.208004:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19730 00000100:00100000:1.0:1789760793.208016:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723314995838 time 723326536468 state 0x2 ref 0 00000100:00100000:2.0:1789760793.212752:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.212828:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503914368 00000100:00100000:3.0:1789760793.212840:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503914368 00000100:00100000:3.0:1789760793.212912:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723314995838 time 723326536468 state 0x0 ref 1 00000100:00100000:3.0:1789760793.212929:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723326796884 time 723326536468 state 0x1 ref 1 00000100:00100000:3.0:1789760793.212958:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723326796884 time 723337565349 state 0x0 ref 1 00000100:00100000:3.0:1789760793.212971:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc12450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503914368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.213033:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80773120, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26858 00010000:00010000:3.0:1789760793.213045:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.213066:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.213071:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.213079:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.213095:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.217372:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987027 is committed 00002000:00010000:3.0:1789760793.217717:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80773120 -> 80777216 00002000:00010000:3.0:1789760793.217723:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157784 -> 157792 00010000:00010000:3.0:1789760793.217738:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.217749:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.217755:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.217760:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.217764:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.217772:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.217793:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.217798:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2341 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80773120->80777215] (req 80773120->80777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.217968:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc12450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503914368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4990us (5221us total) trans 4294987027 rc 0/0 00000100:00100000:3.0:1789760793.217982:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19731 00000100:00100000:3.0:1789760793.217996:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723326796884 time 723337565349 state 0x2 ref 0 00000100:00100000:2.0:1789760793.220920:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.221096:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503914496 00000100:00100000:0.0:1789760793.221109:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503914496 00000100:00100000:0.0:1789760793.221185:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723326796884 time 723337565349 state 0x0 ref 1 00000100:00100000:0.0:1789760793.221265:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723337825765 time 723337565349 state 0x1 ref 1 00000100:00100000:0.0:1789760793.221304:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723337825765 time 723345909896 state 0x0 ref 1 00000100:00100000:0.0:1789760793.221318:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503914496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.221681:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80777216, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26860 00010000:00010000:0.0:1789760793.221695:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.221715:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.221720:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.221728:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.221744:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.226095:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987028 is committed 00002000:00010000:0.0:1789760793.226325:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80777216 -> 80781312 00002000:00010000:0.0:1789760793.226332:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157792 -> 157800 00010000:00010000:0.0:1789760793.226349:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226364:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226376:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226385:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.226422:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226434:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226459:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.226468:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2348 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80777216->80781311] (req 80777216->80781311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.226661:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503914496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5337us (5749us total) trans 4294987028 rc 0/0 00000100:00100000:0.0:1789760793.226675:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19732 00000100:00100000:0.0:1789760793.226687:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723337825765 time 723345909896 state 0x2 ref 0 00000100:00100000:2.0:1789760793.231377:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.231584:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503914624 00000100:00100000:0.0:1789760793.231598:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503914624 00000100:00100000:0.0:1789760793.231669:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723337825765 time 723345909896 state 0x0 ref 1 00000100:00100000:0.0:1789760793.231688:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723346170312 time 723345909896 state 0x1 ref 1 00000100:00100000:0.0:1789760793.231753:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723346170312 time 723356358659 state 0x0 ref 1 00000100:00100000:0.0:1789760793.231768:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503914624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.231832:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80781312, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26868 00010000:00010000:0.0:1789760793.231843:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.231862:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.231867:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.231875:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.231892:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.236736:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987029 is committed 00002000:00010000:0.0:1789760793.236861:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80781312 -> 80785408 00002000:00010000:0.0:1789760793.236867:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157800 -> 157808 00010000:00010000:0.0:1789760793.236883:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.236899:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.236909:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.236924:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.236930:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.236942:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.236967:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.236976:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f234f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80781312->80785407] (req 80781312->80785407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.237093:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503914624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5320us (5722us total) trans 4294987029 rc 0/0 00000100:00100000:0.0:1789760793.237106:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19733 00000100:00100000:0.0:1789760793.237119:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723346170312 time 723356358659 state 0x2 ref 0 00000100:00100000:2.0:1789760793.241075:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.241196:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503914752 00000100:00100000:1.0:1789760793.241263:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503914752 00000100:00100000:1.0:1789760793.241813:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723346170312 time 723356358659 state 0x0 ref 1 00000100:00100000:1.0:1789760793.241833:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723356619075 time 723356358659 state 0x1 ref 1 00000100:00100000:1.0:1789760793.241902:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723356619075 time 723366507962 state 0x0 ref 1 00000100:00100000:1.0:1789760793.241918:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503914752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.241985:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80785408, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26870 00010000:00010000:1.0:1789760793.241997:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.242017:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.242022:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.242030:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.242052:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.247105:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987030 is committed 00002000:00010000:3.0:1789760793.247367:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80785408 -> 80789504 00002000:00010000:3.0:1789760793.247375:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157808 -> 157816 00010000:00010000:3.0:1789760793.247489:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.247506:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.247516:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.247587:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.247593:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.247671:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.247695:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.247704:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2356 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80785408->80789503] (req 80785408->80789503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.247928:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503914752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6006us (6864us total) trans 4294987030 rc 0/0 00000100:00100000:2.0:1789760793.247941:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19734 00000100:00100000:2.0:1789760793.247954:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723356619075 time 723366507962 state 0x2 ref 0 00000100:00100000:2.0:1789760793.252272:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.252364:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503914880 00000100:00100000:3.0:1789760793.252378:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503914880 00000100:00100000:3.0:1789760793.252478:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723356619075 time 723366507962 state 0x0 ref 1 00000100:00100000:3.0:1789760793.252509:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723366768378 time 723366507962 state 0x1 ref 1 00000100:00100000:3.0:1789760793.252562:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723366768378 time 723377158684 state 0x0 ref 1 00000100:00100000:3.0:1789760793.252577:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503914880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.252644:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80789504, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26878 00010000:00010000:3.0:1789760793.252657:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.252679:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.252685:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.252693:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.252710:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.265980:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987031 is committed 00002000:00010000:3.0:1789760793.266115:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80789504 -> 80793600 00002000:00010000:3.0:1789760793.266124:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157816 -> 157824 00010000:00010000:3.0:1789760793.266143:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.266161:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.266171:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.266180:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.266186:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.266195:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.266423:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.266434:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f235d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80789504->80793599] (req 80789504->80793599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.267321:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503914880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14735us (15052us total) trans 4294987031 rc 0/0 00000100:00100000:3.0:1789760793.267338:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19735 00000100:00100000:3.0:1789760793.267353:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723366768378 time 723377158684 state 0x2 ref 0 00000100:00100000:0.0:1789760793.270717:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.270955:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915008 00000100:00100000:2.0:1789760793.270968:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915008 00000100:00100000:2.0:1789760793.271042:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723366768378 time 723377158684 state 0x0 ref 1 00000100:00100000:2.0:1789760793.271060:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723377419100 time 723377158684 state 0x1 ref 1 00000100:00100000:2.0:1789760793.271102:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723377419100 time 723395708457 state 0x0 ref 1 00000100:00100000:2.0:1789760793.271117:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503915008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.271178:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80793600, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26880 00010000:00010000:2.0:1789760793.271189:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.271218:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.271223:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.271231:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.271248:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.279876:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987032 is committed 00002000:00010000:0.0:1789760793.280026:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80793600 -> 80797696 00002000:00010000:0.0:1789760793.280085:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157824 -> 157832 00010000:00010000:0.0:1789760793.280102:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.280112:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.280120:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.280125:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.280130:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.280137:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.280161:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.280169:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2364 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80793600->80797695] (req 80793600->80797695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.280542:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503915008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9419us (9831us total) trans 4294987032 rc 0/0 00000100:00100000:0.0:1789760793.280555:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19736 00000100:00100000:0.0:1789760793.280568:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723377419100 time 723395708457 state 0x2 ref 0 00000100:00100000:2.0:1789760793.284268:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.284866:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915136 00000100:00100000:0.0:1789760793.284882:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915136 00000100:00100000:1.0:1789760793.285052:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723377419100 time 723395708457 state 0x0 ref 1 00000100:00100000:1.0:1789760793.285072:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723395968873 time 723395708457 state 0x1 ref 1 00000100:00100000:1.0:1789760793.285132:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723395968873 time 723409739489 state 0x0 ref 1 00000100:00100000:1.0:1789760793.285147:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.285219:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80797696, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26888 00010000:00010000:1.0:1789760793.285231:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.285253:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.285259:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.285266:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.285283:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.293999:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987033 is committed 00002000:00010000:2.0:1789760793.294343:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80797696 -> 80801792 00002000:00010000:2.0:1789760793.294352:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157832 -> 157840 00010000:00010000:2.0:1789760793.294372:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.294568:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.294582:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.294591:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.294596:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.294607:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.294634:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.294643:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f236b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80797696->80801791] (req 80797696->80801791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.294833:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9682us (10573us total) trans 4294987033 rc 0/0 00000100:00100000:2.0:1789760793.294846:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19737 00000100:00100000:2.0:1789760793.294860:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723395968873 time 723409739489 state 0x2 ref 0 00000100:00100000:2.0:1789760793.301088:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.301272:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915264 00000100:00100000:1.0:1789760793.301286:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915264 00000100:00100000:1.0:1789760793.301368:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723395968873 time 723409739489 state 0x0 ref 1 00000100:00100000:1.0:1789760793.301387:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723409999905 time 723409739489 state 0x1 ref 1 00000100:00100000:1.0:1789760793.301625:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723409999905 time 723426229522 state 0x0 ref 1 00000100:00100000:1.0:1789760793.301642:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503915264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.301783:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80801792, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26890 00010000:00010000:1.0:1789760793.301795:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.301887:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.301892:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.301901:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.301919:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.308792:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987034 is committed 00002000:00010000:1.0:1789760793.309018:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80801792 -> 80805888 00002000:00010000:1.0:1789760793.309025:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157840 -> 157848 00010000:00010000:1.0:1789760793.309043:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309056:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309072:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309080:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.309284:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309298:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309325:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.309334:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2372 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80801792->80805887] (req 80801792->80805887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.309685:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503915264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8036us (8603us total) trans 4294987034 rc 0/0 00000100:00100000:1.0:1789760793.309697:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19738 00000100:00100000:1.0:1789760793.309720:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723409999905 time 723426229522 state 0x2 ref 0 00000100:00100000:0.0:1789760793.314169:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.314555:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915392 00000100:00100000:3.0:1789760793.314568:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915392 00000100:00100000:3.0:1789760793.314643:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723409999905 time 723426229522 state 0x0 ref 1 00000100:00100000:3.0:1789760793.314662:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723426489938 time 723426229522 state 0x1 ref 1 00000100:00100000:3.0:1789760793.314699:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723426489938 time 723439304950 state 0x0 ref 1 00000100:00100000:3.0:1789760793.314718:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503915392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.314783:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80805888, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26898 00010000:00010000:3.0:1789760793.314793:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314813:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.314819:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.314827:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314844:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.323266:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987035 is committed 00002000:00010000:3.0:1789760793.323556:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80805888 -> 80809984 00002000:00010000:3.0:1789760793.323695:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157848 -> 157856 00010000:00010000:3.0:1789760793.323714:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.323729:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.323738:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.323746:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.323752:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.323760:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.323785:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.323792:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2379 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80805888->80809983] (req 80805888->80809983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.324048:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503915392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9324us (9886us total) trans 4294987035 rc 0/0 00000100:00100000:3.0:1789760793.324061:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19739 00000100:00100000:3.0:1789760793.324074:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723426489938 time 723439304950 state 0x2 ref 0 00000100:00100000:3.0:1789760793.328384:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.328997:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915520 00000100:00100000:0.0:1789760793.329011:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915520 00000100:00100000:0.0:1789760793.329090:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723426489938 time 723439304950 state 0x0 ref 1 00000100:00100000:0.0:1789760793.329110:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723439565366 time 723439304950 state 0x1 ref 1 00000100:00100000:0.0:1789760793.329157:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723439565366 time 723453761888 state 0x0 ref 1 00000100:00100000:0.0:1789760793.329171:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.329244:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80809984, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268a0 00010000:00010000:0.0:1789760793.329256:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.329277:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.329283:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.329291:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.329309:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.336521:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987036 is committed 00002000:00010000:0.0:1789760793.337446:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80809984 -> 80814080 00002000:00010000:0.0:1789760793.337455:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157856 -> 157864 00010000:00010000:0.0:1789760793.337472:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.337499:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.337515:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.337524:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.337531:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.337542:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.337580:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.337588:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2380 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80809984->80814079] (req 80809984->80814079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.337859:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8682us (9480us total) trans 4294987036 rc 0/0 00000100:00100000:0.0:1789760793.337872:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19740 00000100:00100000:0.0:1789760793.337884:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723439565366 time 723453761888 state 0x2 ref 0 00000100:00100000:2.0:1789760793.342524:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.342634:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915648 00000100:00100000:1.0:1789760793.342649:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915648 00000100:00100000:1.0:1789760793.342741:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723439565366 time 723453761888 state 0x0 ref 1 00000100:00100000:1.0:1789760793.342760:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723454022304 time 723453761888 state 0x1 ref 1 00000100:00100000:3.0:1789760793.342830:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723454022304 time 723467432139 state 0x0 ref 1 00000100:00100000:3.0:1789760793.342848:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.342926:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80814080, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268a8 00010000:00010000:3.0:1789760793.342938:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.342960:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.342966:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.342974:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.342992:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.350056:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987037 is committed 00002000:00010000:3.0:1789760793.352617:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80814080 -> 80818176 00002000:00010000:3.0:1789760793.352629:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157864 -> 157872 00010000:00010000:3.0:1789760793.352647:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.352662:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.352673:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.352681:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.352687:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.352698:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.352722:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.352737:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2387 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80814080->80818175] (req 80814080->80818175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.352937:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10085us (10421us total) trans 4294987037 rc 0/0 00000100:00100000:3.0:1789760793.352949:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19741 00000100:00100000:3.0:1789760793.352960:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723454022304 time 723467432139 state 0x2 ref 0 00000100:00100000:2.0:1789760793.357868:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.360326:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915776 00000100:00100000:1.0:1789760793.360350:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915776 00000100:00100000:1.0:1789760793.360457:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723454022304 time 723467432139 state 0x0 ref 1 00000100:00100000:1.0:1789760793.360477:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723467692555 time 723467432139 state 0x1 ref 1 00000100:00100000:1.0:1789760793.360516:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723467692555 time 723485122103 state 0x0 ref 1 00000100:00100000:1.0:1789760793.360533:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.360598:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80818176, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268b0 00010000:00010000:1.0:1789760793.360610:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361004:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.361010:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.361019:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361037:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.367178:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987038 is committed 00002000:00010000:1.0:1789760793.367746:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80818176 -> 80822272 00002000:00010000:1.0:1789760793.367753:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157872 -> 157880 00010000:00010000:1.0:1789760793.367771:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.367792:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.367803:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.367810:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.367816:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.367826:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.367851:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.367858:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f238e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80818176->80822271] (req 80818176->80822271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.368027:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503915776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7491us (10166us total) trans 4294987038 rc 0/0 00000100:00100000:1.0:1789760793.368040:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19742 00000100:00100000:1.0:1789760793.368053:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723467692555 time 723485122103 state 0x2 ref 0 00000100:00100000:1.0:1789760793.372184:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.372427:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503915904 00000100:00100000:2.0:1789760793.372439:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503915904 00000100:00100000:2.0:1789760793.372577:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723467692555 time 723485122103 state 0x0 ref 1 00000100:00100000:2.0:1789760793.372598:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723485382519 time 723485122103 state 0x1 ref 1 00000100:00100000:2.0:1789760793.372633:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723485382519 time 723497239791 state 0x0 ref 1 00000100:00100000:2.0:1789760793.372648:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503915904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.372710:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80822272, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268b8 00010000:00010000:2.0:1789760793.372721:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.372740:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.372745:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.372752:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.372770:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.380829:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987039 is committed 00002000:00010000:2.0:1789760793.381678:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80822272 -> 80826368 00002000:00010000:2.0:1789760793.381688:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157880 -> 157888 00010000:00010000:2.0:1789760793.381707:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.381723:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.381732:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.381740:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.381746:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.381757:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.381785:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.381794:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2395 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80822272->80826367] (req 80822272->80826367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.381971:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503915904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9318us (9792us total) trans 4294987039 rc 0/0 00000100:00100000:2.0:1789760793.381983:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19743 00000100:00100000:2.0:1789760793.381996:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723485382519 time 723497239791 state 0x2 ref 0 00000100:00100000:2.0:1789760793.397800:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.399691:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916032 00000100:00100000:1.0:1789760793.399722:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916032 00000100:00100000:1.0:1789760793.399817:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723485382519 time 723497239791 state 0x0 ref 1 00000100:00100000:1.0:1789760793.399837:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723497500207 time 723497239791 state 0x1 ref 1 00000100:00100000:1.0:1789760793.399884:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723497500207 time 723524489953 state 0x0 ref 1 00000100:00100000:1.0:1789760793.399900:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503916032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.400080:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80826368, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268c0 00010000:00010000:1.0:1789760793.400092:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.400114:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.400120:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.400127:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.400144:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.408976:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987040 is committed 00002000:00010000:1.0:1789760793.409174:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80826368 -> 80830464 00002000:00010000:1.0:1789760793.409180:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157888 -> 157896 00010000:00010000:1.0:1789760793.409211:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.409233:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.409251:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.409259:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.409264:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.409274:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.409539:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.409556:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f239c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80826368->80830463] (req 80826368->80830463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.409677:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503916032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9774us (11884us total) trans 4294987040 rc 0/0 00000100:00100000:1.0:1789760793.409689:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19744 00000100:00100000:1.0:1789760793.409702:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723497500207 time 723524489953 state 0x2 ref 0 00000100:00100000:0.0:1789760793.420638:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.420831:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916160 00000100:00100000:3.0:1789760793.420844:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916160 00000100:00100000:3.0:1789760793.420915:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723497500207 time 723524489953 state 0x0 ref 1 00000100:00100000:3.0:1789760793.420933:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723524750369 time 723524489953 state 0x1 ref 1 00000100:00100000:3.0:1789760793.420969:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723524750369 time 723545575658 state 0x0 ref 1 00000100:00100000:3.0:1789760793.420982:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.421039:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80830464, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268c8 00010000:00010000:3.0:1789760793.421050:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.421070:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.421076:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.421084:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.421099:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.427647:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987041 is committed 00002000:00010000:3.0:1789760793.427825:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80830464 -> 80834560 00002000:00010000:3.0:1789760793.427831:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157896 -> 157904 00010000:00010000:3.0:1789760793.427848:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.427862:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.427873:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.427880:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.427886:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.427896:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.427923:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.427931:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80830464->80834559] (req 80830464->80834559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.428129:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7141us (7496us total) trans 4294987041 rc 0/0 00000100:00100000:3.0:1789760793.428142:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19745 00000100:00100000:3.0:1789760793.428155:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723524750369 time 723545575658 state 0x2 ref 0 00000100:00100000:1.0:1789760793.434164:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.434444:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916288 00000100:00100000:2.0:1789760793.434458:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916288 00000100:00100000:2.0:1789760793.434609:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723524750369 time 723545575658 state 0x0 ref 1 00000100:00100000:2.0:1789760793.434631:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723545836074 time 723545575658 state 0x1 ref 1 00000100:00100000:1.0:1789760793.434757:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723545836074 time 723559359999 state 0x0 ref 1 00000100:00100000:1.0:1789760793.434774:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503916288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.434844:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80834560, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268d0 00010000:00010000:1.0:1789760793.434856:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.434877:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.434883:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.434891:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.434910:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.443898:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987042 is committed 00002000:00010000:1.0:1789760793.444058:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80834560 -> 80838656 00002000:00010000:1.0:1789760793.444065:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157904 -> 157912 00010000:00010000:1.0:1789760793.444085:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.444100:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.444110:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.444124:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.444129:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.444138:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.444163:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.444172:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80834560->80838655] (req 80834560->80838655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.444313:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503916288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9537us (10160us total) trans 4294987042 rc 0/0 00000100:00100000:1.0:1789760793.444326:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19746 00000100:00100000:1.0:1789760793.444339:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723545836074 time 723559359999 state 0x2 ref 0 00000100:00100000:2.0:1789760793.448765:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.448830:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916416 00000100:00100000:1.0:1789760793.448844:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916416 00000100:00100000:1.0:1789760793.448920:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723545836074 time 723559359999 state 0x0 ref 1 00000100:00100000:1.0:1789760793.448939:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723559620415 time 723559359999 state 0x1 ref 1 00000100:00100000:1.0:1789760793.448976:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723559620415 time 723573581985 state 0x0 ref 1 00000100:00100000:1.0:1789760793.448996:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.449059:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80838656, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268d8 00010000:00010000:1.0:1789760793.449069:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.449088:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.449093:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.449102:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.449119:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.454446:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987043 is committed 00002000:00010000:1.0:1789760793.454750:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80838656 -> 80842752 00002000:00010000:1.0:1789760793.454757:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157912 -> 157920 00010000:00010000:1.0:1789760793.454775:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.454791:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.454801:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.454809:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.454816:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.454826:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.454851:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.454858:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80838656->80842751] (req 80838656->80842751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.454982:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5983us (6226us total) trans 4294987043 rc 0/0 00000100:00100000:1.0:1789760793.454994:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19747 00000100:00100000:1.0:1789760793.455006:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723559620415 time 723573581985 state 0x2 ref 0 00000100:00100000:0.0:1789760793.463630:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.463817:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916544 00000100:00100000:1.0:1789760793.463830:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916544 00000100:00100000:0.0:1789760793.463984:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723559620415 time 723573581985 state 0x0 ref 1 00000100:00100000:0.0:1789760793.464008:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723573842401 time 723573581985 state 0x1 ref 1 00000100:00100000:0.0:1789760793.464043:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723573842401 time 723588649471 state 0x0 ref 1 00000100:00100000:0.0:1789760793.464056:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503916544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.464121:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80842752, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268e0 00010000:00010000:0.0:1789760793.464131:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.464151:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.464156:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.464163:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.464179:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.471868:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987044 is committed 00002000:00010000:0.0:1789760793.475326:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80842752 -> 80846848 00002000:00010000:0.0:1789760793.475341:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157920 -> 157928 00010000:00010000:0.0:1789760793.475360:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.475377:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.475387:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.475413:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.475420:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.475431:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.475457:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.475465:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f23b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80842752->80846847] (req 80842752->80846847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.475765:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503916544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11702us (12142us total) trans 4294987044 rc 0/0 00000100:00100000:0.0:1789760793.475778:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19748 00000100:00100000:0.0:1789760793.475790:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723573842401 time 723588649471 state 0x2 ref 0 00000100:00100000:3.0:1789760793.486700:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.488749:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916672 00000100:00100000:1.0:1789760793.488770:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916672 00000100:00100000:2.0:1789760793.488997:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723573842401 time 723588649471 state 0x0 ref 1 00000100:00100000:2.0:1789760793.489017:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723588909887 time 723588649471 state 0x1 ref 1 00000100:00100000:2.0:1789760793.489097:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723588909887 time 723613703032 state 0x0 ref 1 00000100:00100000:2.0:1789760793.489114:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.489182:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80846848, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268e8 00010000:00010000:2.0:1789760793.489193:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.489588:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.489596:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.489605:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.489625:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.495697:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987045 is committed 00002000:00010000:2.0:1789760793.495849:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80846848 -> 80850944 00002000:00010000:2.0:1789760793.495855:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157928 -> 157936 00010000:00010000:2.0:1789760793.495872:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495888:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495898:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495906:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.495911:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495922:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495946:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495954:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f23bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80846848->80850943] (req 80846848->80850943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.496124:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7008us (9432us total) trans 4294987045 rc 0/0 00000100:00100000:2.0:1789760793.496137:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19749 00000100:00100000:2.0:1789760793.496149:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723588909887 time 723613703032 state 0x2 ref 0 00000100:00100000:2.0:1789760793.501204:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.501355:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916800 00000100:00100000:1.0:1789760793.501371:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916800 00000100:00100000:1.0:1789760793.501495:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723588909887 time 723613703032 state 0x0 ref 1 00000100:00100000:1.0:1789760793.501515:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723613963448 time 723613703032 state 0x1 ref 1 00000100:00100000:0.0:1789760793.501752:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723613963448 time 723626355625 state 0x0 ref 1 00000100:00100000:0.0:1789760793.501771:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503916800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.501831:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80850944, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268f0 00010000:00010000:0.0:1789760793.501843:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.501869:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.501874:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.501881:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.501897:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.509064:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987046 is committed 00002000:00010000:0.0:1789760793.510760:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80850944 -> 80855040 00002000:00010000:0.0:1789760793.510773:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157936 -> 157944 00010000:00010000:0.0:1789760793.510792:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.510807:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.510817:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.510825:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.510831:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.510840:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.510870:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.510878:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f23c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80850944->80855039] (req 80850944->80855039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.511016:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503916800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9243us (9828us total) trans 4294987046 rc 0/0 00000100:00100000:0.0:1789760793.511030:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19750 00000100:00100000:0.0:1789760793.511043:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723613963448 time 723626355625 state 0x2 ref 0 00000100:00100000:0.0:1789760793.517043:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.517289:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503916928 00000100:00100000:1.0:1789760793.517314:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503916928 00000100:00100000:1.0:1789760793.517429:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723613963448 time 723626355625 state 0x0 ref 1 00000100:00100000:1.0:1789760793.517450:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723626616041 time 723626355625 state 0x1 ref 1 00000100:00100000:1.0:1789760793.517495:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723626616041 time 723642101033 state 0x0 ref 1 00000100:00100000:1.0:1789760793.517511:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.517583:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80855040, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x268f8 00010000:00010000:1.0:1789760793.517593:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.517612:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.517617:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.517625:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.517641:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.523915:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987047 is committed 00002000:00010000:1.0:1789760793.524082:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80855040 -> 80859136 00002000:00010000:1.0:1789760793.524089:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157944 -> 157952 00010000:00010000:1.0:1789760793.524105:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.524120:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.524136:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.524143:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.524149:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.524160:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.524186:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.524195:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80855040->80859135] (req 80855040->80859135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.524333:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503916928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6819us (7297us total) trans 4294987047 rc 0/0 00000100:00100000:1.0:1789760793.524345:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19751 00000100:00100000:1.0:1789760793.524357:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723626616041 time 723642101033 state 0x2 ref 0 00000100:00100000:2.0:1789760793.527900:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.527997:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917056 00000100:00100000:1.0:1789760793.528010:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917056 00000100:00100000:1.0:1789760793.528084:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723626616041 time 723642101033 state 0x0 ref 1 00000100:00100000:1.0:1789760793.528102:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723642361449 time 723642101033 state 0x1 ref 1 00000100:00100000:1.0:1789760793.528143:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723642361449 time 723652749353 state 0x0 ref 1 00000100:00100000:1.0:1789760793.528153:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503917056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.528211:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80859136, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26900 00010000:00010000:1.0:1789760793.528220:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.528236:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.528240:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.528246:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.528259:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.540570:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987048 is committed 00002000:00010000:1.0:1789760793.540786:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80859136 -> 80863232 00002000:00010000:1.0:1789760793.540793:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157952 -> 157960 00010000:00010000:1.0:1789760793.540811:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540825:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540833:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540847:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.540853:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540863:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540886:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540895:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80859136->80863231] (req 80859136->80863231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.541009:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503917056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12847us (13117us total) trans 4294987048 rc 0/0 00000100:00100000:1.0:1789760793.541021:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19752 00000100:00100000:1.0:1789760793.541034:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723642361449 time 723652749353 state 0x2 ref 0 00000100:00100000:2.0:1789760793.545328:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.546183:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917184 00000100:00100000:3.0:1789760793.546200:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917184 00000100:00100000:3.0:1789760793.546289:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723642361449 time 723652749353 state 0x0 ref 1 00000100:00100000:3.0:1789760793.546308:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723653009769 time 723652749353 state 0x1 ref 1 00000100:00100000:3.0:1789760793.546345:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723653009769 time 723670951035 state 0x0 ref 1 00000100:00100000:3.0:1789760793.546359:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503917184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.546458:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80863232, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26908 00010000:00010000:3.0:1789760793.546470:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.546502:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.546507:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.546516:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.546692:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.553325:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987049 is committed 00002000:00010000:3.0:1789760793.553475:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80863232 -> 80867328 00002000:00010000:3.0:1789760793.553490:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157960 -> 157968 00010000:00010000:3.0:1789760793.553508:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.553532:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.553541:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.553550:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.553555:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.553567:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.553592:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.553607:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80863232->80867327] (req 80863232->80867327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.553745:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503917184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7381us (8426us total) trans 4294987049 rc 0/0 00000100:00100000:3.0:1789760793.553757:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19753 00000100:00100000:3.0:1789760793.553769:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723653009769 time 723670951035 state 0x2 ref 0 00000100:00100000:2.0:1789760793.558375:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.558720:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917312 00000100:00100000:1.0:1789760793.558734:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917312 00000100:00100000:1.0:1789760793.558810:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723653009769 time 723670951035 state 0x0 ref 1 00000100:00100000:1.0:1789760793.558830:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723671211451 time 723670951035 state 0x1 ref 1 00000100:00100000:1.0:1789760793.558867:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723671211451 time 723683472727 state 0x0 ref 1 00000100:00100000:1.0:1789760793.558884:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503917312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.558946:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80867328, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26910 00010000:00010000:1.0:1789760793.558957:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.558978:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.558984:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.558992:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.559008:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.567096:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987050 is committed 00002000:00010000:0.0:1789760793.567357:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80867328 -> 80871424 00002000:00010000:0.0:1789760793.567364:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157968 -> 157976 00010000:00010000:0.0:1789760793.567381:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.568757:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.568834:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.568843:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.568849:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.568862:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.568890:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.568899:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f23e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80867328->80871423] (req 80867328->80871423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.569082:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503917312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10191us (10713us total) trans 4294987050 rc 0/0 00000100:00100000:0.0:1789760793.569094:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19754 00000100:00100000:0.0:1789760793.569107:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723671211451 time 723683472727 state 0x2 ref 0 00000100:00100000:2.0:1789760793.575843:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.575956:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917440 00000100:00100000:1.0:1789760793.575969:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917440 00000100:00100000:1.0:1789760793.576049:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723671211451 time 723683472727 state 0x0 ref 1 00000100:00100000:1.0:1789760793.576070:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723683733143 time 723683472727 state 0x1 ref 1 00000100:00100000:1.0:1789760793.576108:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723683733143 time 723700713831 state 0x0 ref 1 00000100:00100000:1.0:1789760793.576122:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503917440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.576186:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80871424, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26918 00010000:00010000:1.0:1789760793.576198:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.576967:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.576973:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.576982:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.577001:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.585074:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987051 is committed 00002000:00010000:1.0:1789760793.585344:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80871424 -> 80875520 00002000:00010000:1.0:1789760793.585351:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157976 -> 157984 00010000:00010000:1.0:1789760793.585370:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.585384:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.585493:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.585502:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.585508:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.585592:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.585625:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.585634:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f23e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80871424->80875519] (req 80871424->80875519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.585869:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503917440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9742us (10033us total) trans 4294987051 rc 0/0 00000100:00100000:1.0:1789760793.585880:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19755 00000100:00100000:1.0:1789760793.585892:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723683733143 time 723700713831 state 0x2 ref 0 00000100:00100000:2.0:1789760793.589076:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.589196:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917568 00000100:00100000:1.0:1789760793.589210:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917568 00000100:00100000:1.0:1789760793.589283:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723683733143 time 723700713831 state 0x0 ref 1 00000100:00100000:1.0:1789760793.589298:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723700974247 time 723700713831 state 0x1 ref 1 00000100:00100000:1.0:1789760793.589328:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723700974247 time 723713933418 state 0x0 ref 1 00000100:00100000:1.0:1789760793.589342:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503917568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.589430:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80875520, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26920 00010000:00010000:1.0:1789760793.589443:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.589470:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.589489:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.589497:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.589514:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.597275:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987052 is committed 00002000:00010000:1.0:1789760793.597768:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80875520 -> 80879616 00002000:00010000:1.0:1789760793.597774:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157984 -> 157992 00010000:00010000:1.0:1789760793.597792:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.597807:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.597817:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.597824:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.597831:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.597840:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.597866:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.597881:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f23f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80875520->80879615] (req 80875520->80879615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.598005:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503917568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8658us (8942us total) trans 4294987052 rc 0/0 00000100:00100000:1.0:1789760793.598018:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19756 00000100:00100000:1.0:1789760793.598030:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723700974247 time 723713933418 state 0x2 ref 0 00000100:00100000:2.0:1789760793.610635:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.612515:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917696 00000100:00100000:3.0:1789760793.612535:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917696 00000100:00100000:3.0:1789760793.612701:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723700974247 time 723713933418 state 0x0 ref 1 00000100:00100000:3.0:1789760793.612721:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723714193834 time 723713933418 state 0x1 ref 1 00000100:00100000:3.0:1789760793.612765:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723714193834 time 723737370888 state 0x0 ref 1 00000100:00100000:3.0:1789760793.612782:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503917696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.612848:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80879616, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26928 00010000:00010000:3.0:1789760793.612860:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.612881:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.612887:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.612895:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.612919:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.619149:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987053 is committed 00002000:00010000:3.0:1789760793.619371:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80879616 -> 80883712 00002000:00010000:3.0:1789760793.619377:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 157992 -> 158000 00010000:00010000:3.0:1789760793.619411:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.619427:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.619437:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.619446:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.619451:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.619461:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.619496:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.619505:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f23f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80879616->80883711] (req 80879616->80883711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.619835:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503917696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7048us (9206us total) trans 4294987053 rc 0/0 00000100:00100000:3.0:1789760793.619849:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19757 00000100:00100000:3.0:1789760793.619862:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723714193834 time 723737370888 state 0x2 ref 0 00000100:00100000:2.0:1789760793.625514:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.625966:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917824 00000100:00100000:0.0:1789760793.625978:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917824 00000100:00100000:0.0:1789760793.626055:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723714193834 time 723737370888 state 0x0 ref 1 00000100:00100000:0.0:1789760793.626075:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723737631304 time 723737370888 state 0x1 ref 1 00000100:00100000:0.0:1789760793.626099:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723737631304 time 723750706394 state 0x0 ref 1 00000100:00100000:0.0:1789760793.626113:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503917824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.626175:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80883712, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26930 00010000:00010000:0.0:1789760793.626188:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.626219:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.626224:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.626233:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.626251:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.633461:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987054 is committed 00002000:00010000:0.0:1789760793.633783:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80883712 -> 80887808 00002000:00010000:0.0:1789760793.633790:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158000 -> 158008 00010000:00010000:0.0:1789760793.633809:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.633825:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.633835:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.633844:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.633851:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.633862:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.633896:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.633906:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f23fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80883712->80887807] (req 80883712->80887807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.634070:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503917824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7952us (8563us total) trans 4294987054 rc 0/0 00000100:00100000:0.0:1789760793.634082:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19758 00000100:00100000:0.0:1789760793.634094:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723737631304 time 723750706394 state 0x2 ref 0 00000100:00100000:2.0:1789760793.638221:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.638540:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503917952 00000100:00100000:1.0:1789760793.638553:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503917952 00000100:00100000:1.0:1789760793.638701:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723737631304 time 723750706394 state 0x0 ref 1 00000100:00100000:1.0:1789760793.638719:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723750966810 time 723750706394 state 0x1 ref 1 00000100:00100000:1.0:1789760793.638758:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723750966810 time 723763364156 state 0x0 ref 1 00000100:00100000:1.0:1789760793.638773:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503917952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.638838:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80887808, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26938 00010000:00010000:1.0:1789760793.639019:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.639037:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.639042:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.639119:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.639140:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.645431:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987055 is committed 00002000:00010000:1.0:1789760793.646047:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80887808 -> 80891904 00002000:00010000:1.0:1789760793.646056:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158008 -> 158016 00010000:00010000:1.0:1789760793.646073:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.646088:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.646099:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.646107:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.646113:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.646123:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.646149:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.646157:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2405 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80887808->80891903] (req 80887808->80891903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.646348:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503917952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7571us (8146us total) trans 4294987055 rc 0/0 00000100:00100000:1.0:1789760793.646361:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19759 00000100:00100000:1.0:1789760793.646374:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723750966810 time 723763364156 state 0x2 ref 0 00000100:00100000:2.0:1789760793.650724:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.650813:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918080 00000100:00100000:1.0:1789760793.650827:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918080 00000100:00100000:1.0:1789760793.650898:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723750966810 time 723763364156 state 0x0 ref 1 00000100:00100000:1.0:1789760793.650917:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723763624572 time 723763364156 state 0x1 ref 1 00000100:00100000:1.0:1789760793.650954:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723763624572 time 723775560480 state 0x0 ref 1 00000100:00100000:1.0:1789760793.650968:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.651034:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80891904, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26940 00010000:00010000:1.0:1789760793.651045:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.651065:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.651070:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.651077:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.651093:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.656295:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987056 is committed 00002000:00010000:1.0:1789760793.656550:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80891904 -> 80896000 00002000:00010000:1.0:1789760793.656556:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158016 -> 158024 00010000:00010000:1.0:1789760793.656640:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.656657:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.656668:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.656677:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.656682:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.656693:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.656719:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.656729:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f240c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80891904->80895999] (req 80891904->80895999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.656891:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5917us (6175us total) trans 4294987056 rc 0/0 00000100:00100000:1.0:1789760793.656906:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19760 00000100:00100000:1.0:1789760793.656920:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723763624572 time 723775560480 state 0x2 ref 0 00000100:00100000:2.0:1789760793.661324:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.661763:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918208 00000100:00100000:3.0:1789760793.661776:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918208 00000100:00100000:3.0:1789760793.661851:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723763624572 time 723775560480 state 0x0 ref 1 00000100:00100000:3.0:1789760793.661870:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723775820896 time 723775560480 state 0x1 ref 1 00000100:00100000:3.0:1789760793.661936:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723775820896 time 723786541657 state 0x0 ref 1 00000100:00100000:3.0:1789760793.661951:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503918208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.662012:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80896000, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26948 00010000:00010000:3.0:1789760793.662025:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.662045:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.662050:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.662058:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.662076:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.668313:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987057 is committed 00002000:00010000:3.0:1789760793.673165:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80896000 -> 80900096 00002000:00010000:3.0:1789760793.673178:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158024 -> 158032 00010000:00010000:3.0:1789760793.673197:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.673273:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.673284:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.673292:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.673297:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.673309:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.673336:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.673344:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2413 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80896000->80900095] (req 80896000->80900095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.673818:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503918208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11859us (12500us total) trans 4294987057 rc 0/0 00000100:00100000:3.0:1789760793.673832:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19761 00000100:00100000:3.0:1789760793.673845:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723775820896 time 723786541657 state 0x2 ref 0 00000100:00100000:2.0:1789760793.679033:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.679173:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918336 00000100:00100000:0.0:1789760793.679186:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918336 00000100:00100000:0.0:1789760793.679263:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723775820896 time 723786541657 state 0x0 ref 1 00000100:00100000:0.0:1789760793.679283:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723786802073 time 723786541657 state 0x1 ref 1 00000100:00100000:0.0:1789760793.679370:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723786802073 time 723803975187 state 0x0 ref 1 00000100:00100000:0.0:1789760793.679384:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503918336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.679821:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80900096, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26950 00010000:00010000:0.0:1789760793.679835:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.679855:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.679861:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.679869:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.679886:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.684341:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987058 is committed 00002000:00010000:0.0:1789760793.684785:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80900096 -> 80904192 00002000:00010000:0.0:1789760793.684792:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158032 -> 158040 00010000:00010000:0.0:1789760793.684809:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.684824:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.684834:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.684843:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.684849:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.684860:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.684886:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.684901:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f241a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80900096->80904191] (req 80900096->80904191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.685073:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503918336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5684us (6047us total) trans 4294987058 rc 0/0 00000100:00100000:0.0:1789760793.685086:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19762 00000100:00100000:0.0:1789760793.685099:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723786802073 time 723803975187 state 0x2 ref 0 00000100:00100000:2.0:1789760793.689277:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.689843:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918464 00000100:00100000:1.0:1789760793.689857:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918464 00000100:00100000:1.0:1789760793.689937:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723786802073 time 723803975187 state 0x0 ref 1 00000100:00100000:1.0:1789760793.689957:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723804235603 time 723803975187 state 0x1 ref 1 00000100:00100000:1.0:1789760793.690037:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723804235603 time 723814642604 state 0x0 ref 1 00000100:00100000:1.0:1789760793.690052:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.690119:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80904192, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26958 00010000:00010000:1.0:1789760793.690132:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.690153:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.690159:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.690167:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.690185:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.696906:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987059 is committed 00002000:00010000:2.0:1789760793.697045:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80904192 -> 80908288 00002000:00010000:2.0:1789760793.697052:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158040 -> 158048 00010000:00010000:2.0:1789760793.697070:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.697091:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.697101:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.697109:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.697115:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.697125:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.697150:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.697158:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2421 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80904192->80908287] (req 80904192->80908287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.697343:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7285us (8074us total) trans 4294987059 rc 0/0 00000100:00100000:2.0:1789760793.697356:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19763 00000100:00100000:2.0:1789760793.697369:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723804235603 time 723814642604 state 0x2 ref 0 00000100:00100000:2.0:1789760793.701745:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.701853:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918592 00000100:00100000:1.0:1789760793.701867:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918592 00000100:00100000:1.0:1789760793.701941:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723804235603 time 723814642604 state 0x0 ref 1 00000100:00100000:1.0:1789760793.701961:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723814903020 time 723814642604 state 0x1 ref 1 00000100:00100000:1.0:1789760793.702042:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723814903020 time 723826647650 state 0x0 ref 1 00000100:00100000:1.0:1789760793.702056:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.702120:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80908288, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26960 00010000:00010000:1.0:1789760793.702131:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.702151:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.702157:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.702165:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.702182:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.709123:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987060 is committed 00002000:00010000:1.0:1789760793.709412:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80908288 -> 80912384 00002000:00010000:1.0:1789760793.709419:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158048 -> 158056 00010000:00010000:1.0:1789760793.709437:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.709453:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.710271:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.710289:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.710296:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.710308:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.710334:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.710342:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2428 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80908288->80912383] (req 80908288->80912383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.710850:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8787us (9111us total) trans 4294987060 rc 0/0 00000100:00100000:1.0:1789760793.710863:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19764 00000100:00100000:1.0:1789760793.710876:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723814903020 time 723826647650 state 0x2 ref 0 00000100:00100000:0.0:1789760793.716634:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.716720:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918720 00000100:00100000:3.0:1789760793.716733:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918720 00000100:00100000:3.0:1789760793.716808:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723814903020 time 723826647650 state 0x0 ref 1 00000100:00100000:3.0:1789760793.716827:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723826908066 time 723826647650 state 0x1 ref 1 00000100:00100000:3.0:1789760793.716854:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723826908066 time 723841460556 state 0x0 ref 1 00000100:00100000:3.0:1789760793.716867:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.716932:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80912384, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26968 00010000:00010000:3.0:1789760793.716943:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.716964:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.716969:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.716977:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.716995:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.723306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987061 is committed 00002000:00010000:3.0:1789760793.723500:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80912384 -> 80916480 00002000:00010000:3.0:1789760793.723507:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158056 -> 158064 00010000:00010000:3.0:1789760793.723650:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.723666:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.723675:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.723683:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.723689:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.723699:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.723725:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.723734:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f242f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80912384->80916479] (req 80912384->80916479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.723875:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff56c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7003us (7248us total) trans 4294987061 rc 0/0 00000100:00100000:3.0:1789760793.723887:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19765 00000100:00100000:3.0:1789760793.723901:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723826908066 time 723841460556 state 0x2 ref 0 00000100:00100000:2.0:1789760793.727834:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.727940:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918848 00000100:00100000:0.0:1789760793.727954:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918848 00000100:00100000:0.0:1789760793.728030:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723826908066 time 723841460556 state 0x0 ref 1 00000100:00100000:0.0:1789760793.728049:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723841720972 time 723841460556 state 0x1 ref 1 00000100:00100000:0.0:1789760793.728125:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723841720972 time 723852730476 state 0x0 ref 1 00000100:00100000:0.0:1789760793.728140:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503918848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.728213:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80916480, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26970 00010000:00010000:0.0:1789760793.728224:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.728244:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.728249:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.728258:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.728275:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.733777:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987062 is committed 00002000:00010000:1.0:1789760793.733979:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80916480 -> 80920576 00002000:00010000:1.0:1789760793.733986:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158064 -> 158072 00010000:00010000:1.0:1789760793.734003:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.734017:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.734026:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.734034:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.734039:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.734051:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.734076:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.734085:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2436 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80916480->80920575] (req 80916480->80920575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.734299:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503918848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6153us (6472us total) trans 4294987062 rc 0/0 00000100:00100000:1.0:1789760793.734313:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19766 00000100:00100000:1.0:1789760793.734326:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723841720972 time 723852730476 state 0x2 ref 0 00000100:00100000:2.0:1789760793.743701:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.744087:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503918976 00000100:00100000:1.0:1789760793.744102:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503918976 00000100:00100000:1.0:1789760793.744181:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723841720972 time 723852730476 state 0x0 ref 1 00000100:00100000:1.0:1789760793.744213:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723852990892 time 723852730476 state 0x1 ref 1 00000100:00100000:1.0:1789760793.744247:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723852990892 time 723868853218 state 0x0 ref 1 00000100:00100000:1.0:1789760793.744261:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.744323:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80920576, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26978 00010000:00010000:1.0:1789760793.744335:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.744355:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.744360:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.744367:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.744384:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.749145:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987063 is committed 00002000:00010000:2.0:1789760793.749414:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80920576 -> 80924672 00002000:00010000:2.0:1789760793.749420:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158072 -> 158080 00010000:00010000:2.0:1789760793.749439:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.749454:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.749594:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.749604:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.749610:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.749620:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.749646:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.749654:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f243d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80920576->80924671] (req 80920576->80924671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.749823:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503918976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5557us (6133us total) trans 4294987063 rc 0/0 00000100:00100000:2.0:1789760793.749837:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19767 00000100:00100000:2.0:1789760793.749849:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723852990892 time 723868853218 state 0x2 ref 0 00000100:00100000:0.0:1789760793.753896:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.753991:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503919104 00000100:00100000:2.0:1789760793.754004:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503919104 00000100:00100000:2.0:1789760793.754074:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723852990892 time 723868853218 state 0x0 ref 1 00000100:00100000:2.0:1789760793.754093:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723869113634 time 723868853218 state 0x1 ref 1 00000100:00100000:2.0:1789760793.754172:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723869113634 time 723878778125 state 0x0 ref 1 00000100:00100000:2.0:1789760793.754189:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.754262:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80924672, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26980 00010000:00010000:2.0:1789760793.754274:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.754294:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.754300:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.754307:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.754323:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.760186:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987064 is committed 00002000:00010000:2.0:1789760793.760683:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80924672 -> 80928768 00002000:00010000:2.0:1789760793.760691:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158080 -> 158088 00010000:00010000:2.0:1789760793.760709:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.760723:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.760733:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.760746:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.760752:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.760762:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.760787:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.760796:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2444 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80924672->80928767] (req 80924672->80928767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.760923:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6730us (7036us total) trans 4294987064 rc 0/0 00000100:00100000:2.0:1789760793.760936:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19768 00000100:00100000:2.0:1789760793.760948:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723869113634 time 723878778125 state 0x2 ref 0 00000100:00100000:0.0:1789760793.765508:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.765730:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503919232 00000100:00100000:3.0:1789760793.765744:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503919232 00000100:00100000:3.0:1789760793.765815:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723869113634 time 723878778125 state 0x0 ref 1 00000100:00100000:3.0:1789760793.765834:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723879038541 time 723878778125 state 0x1 ref 1 00000100:00100000:3.0:1789760793.765869:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723879038541 time 723890475643 state 0x0 ref 1 00000100:00100000:3.0:1789760793.765881:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503919232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.765943:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80928768, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26988 00010000:00010000:3.0:1789760793.765953:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.765973:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.765979:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.765986:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.766002:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.772947:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987065 is committed 00002000:00010000:0.0:1789760793.773287:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80928768 -> 80932864 00002000:00010000:0.0:1789760793.773294:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158088 -> 158096 00010000:00010000:0.0:1789760793.773311:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.773326:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.773336:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.773344:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.773350:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.773360:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.773386:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.773420:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f244b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80928768->80932863] (req 80928768->80932863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.773735:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503919232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7847us (8248us total) trans 4294987065 rc 0/0 00000100:00100000:0.0:1789760793.773747:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19769 00000100:00100000:0.0:1789760793.773761:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723879038541 time 723890475643 state 0x2 ref 0 00000100:00100000:1.0:1789760793.778738:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.779116:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503919360 00000100:00100000:0.0:1789760793.779128:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503919360 00000100:00100000:0.0:1789760793.779270:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723879038541 time 723890475643 state 0x0 ref 1 00000100:00100000:0.0:1789760793.779292:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723890736059 time 723890475643 state 0x1 ref 1 00000100:00100000:0.0:1789760793.779385:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723890736059 time 723903990981 state 0x0 ref 1 00000100:00100000:0.0:1789760793.779469:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503919360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.779555:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80932864, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x26990 00010000:00010000:0.0:1789760793.779567:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.779597:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760793.779602:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.779610:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.779631:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.784256:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987066 is committed 00002000:00010000:0.0:1789760793.784627:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80932864 -> 80936960 00002000:00010000:0.0:1789760793.784634:0:8434:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:0.0:1789760793.784637:0:8434:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00002000:00010000:0.0:1789760793.784640:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158096 -> 158104 00010000:00010000:0.0:1789760793.784657:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.784673:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.784683:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.784692:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.784698:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.784708:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.784732:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.784746:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2452 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80932864->80936959] (req 80932864->80936959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.784904:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503919360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5433us (6176us total) trans 4294987066 rc 0/0 00000100:00100000:0.0:1789760793.784915:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19770 00000100:00100000:0.0:1789760793.784927:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723890736059 time 723903990981 state 0x2 ref 0 00000100:00100000:2.0:1789760793.790717:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.790811:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503919488 00000100:00100000:1.0:1789760793.790824:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503919488 00000100:00100000:1.0:1789760793.790899:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723890736059 time 723903990981 state 0x0 ref 1 00000100:00100000:1.0:1789760793.790919:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723904251397 time 723903990981 state 0x1 ref 1 00000100:00100000:1.0:1789760793.790955:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723904251397 time 723915561500 state 0x0 ref 1 00000100:00100000:1.0:1789760793.790970:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.791104:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80936960, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26998 00010000:00010000:1.0:1789760793.791117:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.791136:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.791141:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.791150:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.791167:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.796422:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987067 is committed 00002000:00010000:1.0:1789760793.796756:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80936960 -> 80941056 00002000:00010000:1.0:1789760793.796763:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158104 -> 158112 00010000:00010000:1.0:1789760793.796781:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.796806:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.796816:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.796824:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760793.796830:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.796841:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.796865:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.796874:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2459 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80936960->80941055] (req 80936960->80941055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.797028:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6055us (6319us total) trans 4294987067 rc 0/0 00000100:00100000:1.0:1789760793.797048:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19771 00000100:00100000:1.0:1789760793.797060:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723904251397 time 723915561500 state 0x2 ref 0 00000100:00100000:1.0:1789760793.805726:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.805814:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503919616 00000100:00100000:3.0:1789760793.805828:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503919616 00000100:00100000:3.0:1789760793.805902:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723904251397 time 723915561500 state 0x0 ref 1 00000100:00100000:3.0:1789760793.805921:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723915821916 time 723915561500 state 0x1 ref 1 00000100:00100000:3.0:1789760793.805949:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723915821916 time 723930555566 state 0x0 ref 1 00000100:00100000:3.0:1789760793.805963:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.806029:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80941056, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269a0 00010000:00010000:3.0:1789760793.806116:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.806137:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.806143:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.806151:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.806170:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.810453:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987068 is committed 00002000:00010000:3.0:1789760793.810629:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80941056 -> 80945152 00002000:00010000:3.0:1789760793.810636:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158112 -> 158120 00010000:00010000:3.0:1789760793.810653:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.810668:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.810677:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.810692:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.810698:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.810709:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.810735:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.810744:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2460 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80941056->80945151] (req 80941056->80945151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.810856:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4888us (5138us total) trans 4294987068 rc 0/0 00000100:00100000:3.0:1789760793.810868:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19772 00000100:00100000:3.0:1789760793.810880:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723915821916 time 723930555566 state 0x2 ref 0 00000100:00100000:1.0:1789760793.815845:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.816175:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503919744 00000100:00100000:2.0:1789760793.816187:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503919744 00000100:00100000:2.0:1789760793.816281:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723915821916 time 723930555566 state 0x0 ref 1 00000100:00100000:2.0:1789760793.816299:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723930815982 time 723930555566 state 0x1 ref 1 00000100:00100000:2.0:1789760793.816333:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723930815982 time 723940939800 state 0x0 ref 1 00000100:00100000:2.0:1789760793.816347:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.816435:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80945152, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269a8 00010000:00010000:2.0:1789760793.816446:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.816465:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.816471:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.816487:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.816504:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.822725:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987069 is committed 00002000:00010000:2.0:1789760793.822956:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80945152 -> 80949248 00002000:00010000:2.0:1789760793.822963:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158120 -> 158128 00010000:00010000:2.0:1789760793.822981:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.822997:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.823006:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.823014:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.823019:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.823029:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.823124:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.823132:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2467 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80945152->80949247] (req 80945152->80949247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.823352:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6999us (7514us total) trans 4294987069 rc 0/0 00000100:00100000:2.0:1789760793.823365:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19773 00000100:00100000:2.0:1789760793.823378:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723930815982 time 723940939800 state 0x2 ref 0 00000100:00100000:0.0:1789760793.828583:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.828707:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503919872 00000100:00100000:1.0:1789760793.828720:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503919872 00000100:00100000:1.0:1789760793.828789:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723930815982 time 723940939800 state 0x0 ref 1 00000100:00100000:1.0:1789760793.828808:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723941200216 time 723940939800 state 0x1 ref 1 00000100:00100000:1.0:1789760793.828880:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723941200216 time 723953485738 state 0x0 ref 1 00000100:00100000:1.0:1789760793.828895:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760793.828959:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80949248, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269b0 00010000:00010000:1.0:1789760793.828971:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.828991:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760793.828996:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.829004:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.829021:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.838321:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987070 is committed 00002000:00010000:0.0:1789760793.838490:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80949248 -> 80953344 00002000:00010000:0.0:1789760793.838496:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158128 -> 158136 00010000:00010000:0.0:1789760793.838523:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.838539:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.838549:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.838558:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.838563:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.838573:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.838598:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.838607:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f246e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80949248->80953343] (req 80949248->80953343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.838725:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503919872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9824us (10217us total) trans 4294987070 rc 0/0 00000100:00100000:0.0:1789760793.838738:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19774 00000100:00100000:0.0:1789760793.838750:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723941200216 time 723953485738 state 0x2 ref 0 00000100:00100000:1.0:1789760793.843133:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.843458:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920000 00000100:00100000:2.0:1789760793.843473:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920000 00000100:00100000:2.0:1789760793.843562:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723941200216 time 723953485738 state 0x0 ref 1 00000100:00100000:2.0:1789760793.843580:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723953746154 time 723953485738 state 0x1 ref 1 00000100:00100000:2.0:1789760793.843612:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723953746154 time 723968218113 state 0x0 ref 1 00000100:00100000:2.0:1789760793.843626:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.843695:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80953344, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269b8 00010000:00010000:2.0:1789760793.843707:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.843729:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760793.843735:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.843744:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.843762:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.855056:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987071 is committed 00002000:00010000:3.0:1789760793.855373:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80953344 -> 80957440 00002000:00010000:3.0:1789760793.855379:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158136 -> 158144 00010000:00010000:3.0:1789760793.855422:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.855438:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.855449:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.855457:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.855463:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.855474:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.855510:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.855910:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2475 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80953344->80957439] (req 80953344->80957439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.856049:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12418us (12924us total) trans 4294987071 rc 0/0 00000100:00100000:3.0:1789760793.856061:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19775 00000100:00100000:3.0:1789760793.856073:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723953746154 time 723968218113 state 0x2 ref 0 00000100:00100000:1.0:1789760793.860180:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.860271:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920128 00000100:00100000:3.0:1789760793.860284:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920128 00000100:00100000:3.0:1789760793.860352:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723953746154 time 723968218113 state 0x0 ref 1 00000100:00100000:3.0:1789760793.860370:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723968478529 time 723968218113 state 0x1 ref 1 00000100:00100000:3.0:1789760793.860431:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723968478529 time 723985037626 state 0x0 ref 1 00000100:00100000:3.0:1789760793.860445:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.860624:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80957440, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269c0 00010000:00010000:3.0:1789760793.860635:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.860654:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760793.860659:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.860666:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.860683:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.867462:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987072 is committed 00002000:00010000:3.0:1789760793.867844:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80957440 -> 80961536 00002000:00010000:3.0:1789760793.867851:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158144 -> 158152 00010000:00010000:3.0:1789760793.867867:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.867880:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.867890:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.867903:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760793.867909:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.867919:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.867943:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.867950:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f247c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80957440->80961535] (req 80957440->80961535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.868244:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7793us (8069us total) trans 4294987072 rc 0/0 00000100:00100000:3.0:1789760793.868257:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19776 00000100:00100000:3.0:1789760793.868269:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723968478529 time 723985037626 state 0x2 ref 0 00000100:00100000:0.0:1789760793.876496:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.876569:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920256 00000100:00100000:2.0:1789760793.876583:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920256 00000100:00100000:2.0:1789760793.876673:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723968478529 time 723985037626 state 0x0 ref 1 00000100:00100000:2.0:1789760793.876694:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723985298042 time 723985037626 state 0x1 ref 1 00000100:00100000:2.0:1789760793.876732:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723985298042 time 724001337739 state 0x0 ref 1 00000100:00100000:2.0:1789760793.876747:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503920256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.876809:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80961536, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269c8 00010000:00010000:2.0:1789760793.876820:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.876840:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760793.876846:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.876854:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.876874:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.883088:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987073 is committed 00002000:00010000:2.0:1789760793.883428:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80961536 -> 80965632 00002000:00010000:2.0:1789760793.883435:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158152 -> 158160 00010000:00010000:2.0:1789760793.883454:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.883469:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.883496:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.883503:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.883509:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.883519:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.883544:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.883552:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2483 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80961536->80965631] (req 80961536->80965631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.883678:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503920256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6927us (7201us total) trans 4294987073 rc 0/0 00000100:00100000:2.0:1789760793.883690:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19777 00000100:00100000:2.0:1789760793.883705:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723985298042 time 724001337739 state 0x2 ref 0 00000100:00100000:1.0:1789760793.887972:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.888163:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920384 00000100:00100000:0.0:1789760793.888176:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920384 00000100:00100000:0.0:1789760793.888248:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 723985298042 time 724001337739 state 0x0 ref 1 00000100:00100000:0.0:1789760793.888265:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724001598155 time 724001337739 state 0x1 ref 1 00000100:00100000:0.0:1789760793.888341:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724001598155 time 724012947306 state 0x0 ref 1 00000100:00100000:0.0:1789760793.888356:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.888454:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80965632, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269d0 00010000:00010000:0.0:1789760793.888465:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.888500:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760793.888506:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.888514:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.888538:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.894696:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987074 is committed 00002000:00010000:0.0:1789760793.896685:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80965632 -> 80969728 00002000:00010000:0.0:1789760793.896697:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158160 -> 158168 00010000:00010000:0.0:1789760793.896716:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896732:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896742:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896750:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.896755:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896765:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896796:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896804:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f248a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80965632->80969727] (req 80965632->80969727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.897015:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8654us (9052us total) trans 4294987074 rc 0/0 00000100:00100000:0.0:1789760793.897029:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19778 00000100:00100000:0.0:1789760793.897041:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724001598155 time 724012947306 state 0x2 ref 0 00000100:00100000:1.0:1789760793.901766:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.905626:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920512 00000100:00100000:3.0:1789760793.905647:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920512 00000100:00100000:2.0:1789760793.907771:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724001598155 time 724012947306 state 0x0 ref 1 00000100:00100000:2.0:1789760793.907795:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724013207722 time 724012947306 state 0x1 ref 1 00000100:00100000:2.0:1789760793.907899:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724013207722 time 724032504834 state 0x0 ref 1 00000100:00100000:2.0:1789760793.907917:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503920512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.907983:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80969728, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269d8 00010000:00010000:2.0:1789760793.907995:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.908014:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760793.908019:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.908027:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.908044:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.917699:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987075 is committed 00002000:00010000:2.0:1789760793.919350:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80969728 -> 80973824 00002000:00010000:2.0:1789760793.919363:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158168 -> 158176 00010000:00010000:2.0:1789760793.919382:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.919413:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.919422:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.919436:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.919442:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.919453:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.919492:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.919502:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2491 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80969728->80973823] (req 80969728->80973823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.919768:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503920512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11848us (18008us total) trans 4294987075 rc 0/0 00000100:00100000:2.0:1789760793.919783:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19779 00000100:00100000:2.0:1789760793.919795:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724013207722 time 724032504834 state 0x2 ref 0 00000100:00100000:2.0:1789760793.925434:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760793.925787:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920640 00000100:00100000:3.0:1789760793.925803:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920640 00000100:00100000:3.0:1789760793.925887:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724013207722 time 724032504834 state 0x0 ref 1 00000100:00100000:3.0:1789760793.925909:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724032765250 time 724032504834 state 0x1 ref 1 00000100:00100000:3.0:1789760793.925946:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724032765250 time 724050551924 state 0x0 ref 1 00000100:00100000:3.0:1789760793.925962:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760793.926032:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80973824, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269e0 00010000:00010000:3.0:1789760793.926043:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.926064:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760793.926070:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.926078:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.926095:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.932013:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987076 is committed 00002000:00010000:0.0:1789760793.932459:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80973824 -> 80977920 00002000:00010000:0.0:1789760793.932467:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158176 -> 158184 00010000:00010000:0.0:1789760793.932493:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.932508:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.932517:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.932526:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.932531:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.932547:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.932573:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.932580:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2498 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80973824->80977919] (req 80973824->80977919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.932749:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503920640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6782us (7325us total) trans 4294987076 rc 0/0 00000100:00100000:0.0:1789760793.932762:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19780 00000100:00100000:0.0:1789760793.932774:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724032765250 time 724050551924 state 0x2 ref 0 00000100:00100000:1.0:1789760793.938631:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.938684:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920768 00000100:00100000:0.0:1789760793.938697:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920768 00000100:00100000:0.0:1789760793.938773:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724032765250 time 724050551924 state 0x0 ref 1 00000100:00100000:0.0:1789760793.938793:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724050812340 time 724050551924 state 0x1 ref 1 00000100:00100000:0.0:1789760793.938873:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724050812340 time 724063478152 state 0x0 ref 1 00000100:00100000:0.0:1789760793.938888:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503920768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.938960:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80977920, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269e8 00010000:00010000:0.0:1789760793.938972:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.938993:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760793.938998:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.939006:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.939023:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.944438:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987077 is committed 00002000:00010000:0.0:1789760793.944616:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80977920 -> 80982016 00002000:00010000:0.0:1789760793.944623:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158184 -> 158192 00010000:00010000:0.0:1789760793.944640:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.944664:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.944674:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.944680:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.944684:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.944691:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.944715:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.944721:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f249f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80977920->80982015] (req 80977920->80982015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.944886:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503920768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5994us (6263us total) trans 4294987077 rc 0/0 00000100:00100000:0.0:1789760793.944903:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19781 00000100:00100000:0.0:1789760793.944916:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724050812340 time 724063478152 state 0x2 ref 0 00000100:00100000:1.0:1789760793.952190:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760793.952319:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503920896 00000100:00100000:0.0:1789760793.952332:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503920896 00000100:00100000:0.0:1789760793.952440:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724050812340 time 724063478152 state 0x0 ref 1 00000100:00100000:0.0:1789760793.952459:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724063738568 time 724063478152 state 0x1 ref 1 00000100:00100000:0.0:1789760793.952605:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724063738568 time 724077210045 state 0x0 ref 1 00000100:00100000:0.0:1789760793.952620:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503920896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.952690:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80982016, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269f0 00010000:00010000:0.0:1789760793.952698:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.952715:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760793.952719:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.952724:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.952737:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.956944:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987078 is committed 00002000:00010000:0.0:1789760793.957154:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80982016 -> 80986112 00002000:00010000:0.0:1789760793.957161:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158192 -> 158200 00010000:00010000:0.0:1789760793.957178:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.957193:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.957211:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.957219:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760793.957224:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.957234:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.957259:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.957274:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80982016->80986111] (req 80982016->80986111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.957555:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503920896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4928us (5368us total) trans 4294987078 rc 0/0 00000100:00100000:0.0:1789760793.957567:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19782 00000100:00100000:0.0:1789760793.957579:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724063738568 time 724077210045 state 0x2 ref 0 00000100:00100000:2.0:1789760793.960676:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760793.960777:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921024 00000100:00100000:1.0:1789760793.960791:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921024 00000100:00100000:1.0:1789760793.961135:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724063738568 time 724077210045 state 0x0 ref 1 00000100:00100000:1.0:1789760793.961156:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724077470461 time 724077210045 state 0x1 ref 1 00000100:00100000:0.0:1789760793.961728:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724077470461 time 724086295884 state 0x0 ref 1 00000100:00100000:0.0:1789760793.961751:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760793.961815:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80986112, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x269f8 00010000:00010000:0.0:1789760793.961824:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.961843:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760793.961847:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.961853:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.961870:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.967519:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987079 is committed 00002000:00010000:2.0:1789760793.968513:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80986112 -> 80990208 00002000:00010000:2.0:1789760793.968522:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158200 -> 158208 00010000:00010000:2.0:1789760793.968541:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.968563:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.968571:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.968578:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.968583:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.968593:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.968616:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.968624:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f24ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80986112->80990207] (req 80986112->80990207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.969312:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7556us (8636us total) trans 4294987079 rc 0/0 00000100:00100000:2.0:1789760793.969327:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19783 00000100:00100000:2.0:1789760793.969340:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724077470461 time 724086295884 state 0x2 ref 0 00000100:00100000:1.0:1789760793.974957:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.975023:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921152 00000100:00100000:2.0:1789760793.975038:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921152 00000100:00100000:2.0:1789760793.975123:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724077470461 time 724086295884 state 0x0 ref 1 00000100:00100000:2.0:1789760793.975151:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724086556300 time 724086295884 state 0x1 ref 1 00000100:00100000:2.0:1789760793.975187:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724086556300 time 724099792624 state 0x0 ref 1 00000100:00100000:2.0:1789760793.975209:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.975269:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80990208, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a00 00010000:00010000:2.0:1789760793.975279:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.975300:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760793.975305:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.975312:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.975329:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.980808:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987080 is committed 00002000:00010000:2.0:1789760793.981031:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80990208 -> 80994304 00002000:00010000:2.0:1789760793.981039:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158208 -> 158216 00010000:00010000:2.0:1789760793.981119:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.981135:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.981150:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.981158:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760793.981163:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.981173:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.981308:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.981317:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80990208->80994303] (req 80990208->80994303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.981543:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6331us (6596us total) trans 4294987080 rc 0/0 00000100:00100000:2.0:1789760793.981556:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19784 00000100:00100000:2.0:1789760793.981569:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724086556300 time 724099792624 state 0x2 ref 0 00000100:00100000:1.0:1789760793.989348:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760793.991175:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921280 00000100:00100000:2.0:1789760793.991195:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921280 00000100:00100000:2.0:1789760793.991562:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724086556300 time 724099792624 state 0x0 ref 1 00000100:00100000:2.0:1789760793.991587:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724100053040 time 724099792624 state 0x1 ref 1 00000100:00100000:2.0:1789760793.991681:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724100053040 time 724116286027 state 0x0 ref 1 00000100:00100000:2.0:1789760793.991697:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760793.991760:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80994304, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a08 00010000:00010000:2.0:1789760793.991771:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.991789:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760793.991794:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.991802:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.991818:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.002682:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987081 is committed 00002000:00010000:2.0:1789760794.002889:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80994304 -> 80998400 00002000:00010000:2.0:1789760794.002897:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158216 -> 158224 00010000:00010000:2.0:1789760794.002915:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.002930:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.002941:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.002956:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.002963:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.002973:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.002998:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.003007:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80994304->80998399] (req 80994304->80998399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.003188:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11488us (13845us total) trans 4294987081 rc 0/0 00000100:00100000:2.0:1789760794.003217:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19785 00000100:00100000:2.0:1789760794.003232:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724100053040 time 724116286027 state 0x2 ref 0 00000100:00100000:2.0:1789760794.009203:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.009444:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921408 00000100:00100000:0.0:1789760794.009459:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921408 00000100:00100000:0.0:1789760794.009620:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724100053040 time 724116286027 state 0x0 ref 1 00000100:00100000:0.0:1789760794.009640:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724116546443 time 724116286027 state 0x1 ref 1 00000100:00100000:0.0:1789760794.009675:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724116546443 time 724134281008 state 0x0 ref 1 00000100:00100000:0.0:1789760794.009691:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503921408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.009755:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 80998400, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a10 00010000:00010000:0.0:1789760794.009768:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.009788:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.009794:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.009802:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.009820:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760794.011161:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760794.015224:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987082 is committed 00002000:00010000:0.0:1789760794.015353:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 80998400 -> 81002496 00002000:00010000:0.0:1789760794.015360:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158224 -> 158232 00010000:00010000:0.0:1789760794.015377:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.016002:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.016016:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.016024:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.016151:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.016166:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.016195:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.016204:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [80998400->81002495] (req 80998400->81002495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.016349:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503921408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6653us (7152us total) trans 4294987082 rc 0/0 00000100:00100000:0.0:1789760794.016363:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19786 00000100:00100000:0.0:1789760794.016376:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724116546443 time 724134281008 state 0x2 ref 0 00000100:00100000:1.0:1789760794.022676:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.023305:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921536 00000100:00100000:0.0:1789760794.023321:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921536 00000100:00100000:0.0:1789760794.023449:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724116546443 time 724134281008 state 0x0 ref 1 00000100:00100000:0.0:1789760794.023469:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724134541424 time 724134281008 state 0x1 ref 1 00000100:00100000:2.0:1789760794.024124:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724134541424 time 724148725118 state 0x0 ref 1 00000100:00100000:2.0:1789760794.024147:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503921536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.024281:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81002496, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a18 00010000:00010000:2.0:1789760794.024293:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.024385:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.024420:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.024428:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.024446:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.032149:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987083 is committed 00002000:00010000:2.0:1789760794.032779:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81002496 -> 81006592 00002000:00010000:2.0:1789760794.032787:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158232 -> 158240 00010000:00010000:2.0:1789760794.032806:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.032821:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.032831:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.032839:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.032845:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.032855:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.032929:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.032940:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81002496->81006591] (req 81002496->81006591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.033121:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503921536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8974us (10453us total) trans 4294987083 rc 0/0 00000100:00100000:2.0:1789760794.033132:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19787 00000100:00100000:2.0:1789760794.033144:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724134541424 time 724148725118 state 0x2 ref 0 00000100:00100000:1.0:1789760794.037289:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.037492:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921664 00000100:00100000:2.0:1789760794.037507:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921664 00000100:00100000:2.0:1789760794.037579:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724134541424 time 724148725118 state 0x0 ref 1 00000100:00100000:2.0:1789760794.037599:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724148985534 time 724148725118 state 0x1 ref 1 00000100:00100000:2.0:1789760794.037642:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724148985534 time 724162248083 state 0x0 ref 1 00000100:00100000:2.0:1789760794.037656:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc12050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503921664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.037718:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81006592, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a20 00010000:00010000:2.0:1789760794.037729:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.037749:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.037755:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.037763:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.037781:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.044996:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987084 is committed 00002000:00010000:2.0:1789760794.045249:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81006592 -> 81010688 00002000:00010000:2.0:1789760794.045256:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158240 -> 158248 00010000:00010000:2.0:1789760794.045274:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.045290:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.045300:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.045308:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.045315:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.045325:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.045351:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.045365:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f24d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81006592->81010687] (req 81006592->81010687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.045569:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc12050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503921664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7907us (8286us total) trans 4294987084 rc 0/0 00000100:00100000:2.0:1789760794.045584:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19788 00000100:00100000:2.0:1789760794.045598:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724148985534 time 724162248083 state 0x2 ref 0 00000100:00100000:3.0:1789760794.051984:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.052093:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921792 00000100:00100000:0.0:1789760794.052106:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921792 00000100:00100000:0.0:1789760794.052183:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724148985534 time 724162248083 state 0x0 ref 1 00000100:00100000:0.0:1789760794.052213:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724162508499 time 724162248083 state 0x1 ref 1 00000100:00100000:0.0:1789760794.052247:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724162508499 time 724176853397 state 0x0 ref 1 00000100:00100000:0.0:1789760794.052261:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.052320:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81010688, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a28 00010000:00010000:0.0:1789760794.052329:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.052452:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.052458:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.052466:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.052491:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.066307:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987085 is committed 00002000:00010000:0.0:1789760794.066710:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81010688 -> 81014784 00002000:00010000:0.0:1789760794.066716:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158248 -> 158256 00010000:00010000:0.0:1789760794.066734:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.066756:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.066766:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.066773:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.066779:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.066788:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.066813:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.066821:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81010688->81014783] (req 81010688->81014783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.067009:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503921792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14743us (15031us total) trans 4294987085 rc 0/0 00000100:00100000:0.0:1789760794.067022:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19789 00000100:00100000:0.0:1789760794.067087:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724162508499 time 724176853397 state 0x2 ref 0 00000100:00100000:1.0:1789760794.071054:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.071158:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503921920 00000100:00100000:3.0:1789760794.071171:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503921920 00000100:00100000:3.0:1789760794.071242:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724162508499 time 724176853397 state 0x0 ref 1 00000100:00100000:3.0:1789760794.071261:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724177113813 time 724176853397 state 0x1 ref 1 00000100:00100000:3.0:1789760794.071288:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724177113813 time 724195894529 state 0x0 ref 1 00000100:00100000:3.0:1789760794.071303:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503921920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.071360:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81014784, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a30 00010000:00010000:3.0:1789760794.071371:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.072205:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760794.072216:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.072226:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.072246:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.082023:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987086 is committed 00002000:00010000:3.0:1789760794.082229:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81014784 -> 81018880 00002000:00010000:3.0:1789760794.082236:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158256 -> 158264 00010000:00010000:3.0:1789760794.082253:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.082268:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.082278:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.082292:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.082298:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.082309:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.082334:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.082343:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f24de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81014784->81018879] (req 81014784->81018879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.082844:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503921920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11535us (11792us total) trans 4294987086 rc 0/0 00000100:00100000:2.0:1789760794.082858:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19790 00000100:00100000:2.0:1789760794.082870:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724177113813 time 724195894529 state 0x2 ref 0 00000100:00100000:1.0:1789760794.087142:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.088222:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922048 00000100:00100000:2.0:1789760794.088247:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922048 00000100:00100000:2.0:1789760794.088334:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724177113813 time 724195894529 state 0x0 ref 1 00000100:00100000:2.0:1789760794.088353:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724196154945 time 724195894529 state 0x1 ref 1 00000100:00100000:2.0:1789760794.088528:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724196154945 time 724213131844 state 0x0 ref 1 00000100:00100000:2.0:1789760794.088545:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503922048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.088620:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81018880, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a38 00010000:00010000:2.0:1789760794.088632:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.088653:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.088658:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.089145:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.089168:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.095776:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987087 is committed 00002000:00010000:2.0:1789760794.095927:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81018880 -> 81022976 00002000:00010000:2.0:1789760794.095934:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158264 -> 158272 00010000:00010000:2.0:1789760794.095952:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.095965:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.095973:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.095981:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.095986:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.095995:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.096024:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.096032:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81018880->81022975] (req 81018880->81022975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.096949:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503922048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8394us (9813us total) trans 4294987087 rc 0/0 00000100:00100000:2.0:1789760794.096965:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19791 00000100:00100000:2.0:1789760794.096977:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724196154945 time 724213131844 state 0x2 ref 0 00000100:00100000:2.0:1789760794.107279:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.107458:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922176 00000100:00100000:0.0:1789760794.107471:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922176 00000100:00100000:0.0:1789760794.107603:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724196154945 time 724213131844 state 0x0 ref 1 00000100:00100000:0.0:1789760794.107622:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724213392260 time 724213131844 state 0x1 ref 1 00000100:00100000:0.0:1789760794.107684:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724213392260 time 724232289393 state 0x0 ref 1 00000100:00100000:0.0:1789760794.107698:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.107758:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81022976, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a40 00010000:00010000:0.0:1789760794.107768:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.107787:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.107792:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.107799:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.107815:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.117112:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987088 is committed 00002000:00010000:2.0:1789760794.118290:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81022976 -> 81027072 00002000:00010000:2.0:1789760794.118299:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158272 -> 158280 00010000:00010000:2.0:1789760794.118317:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.118332:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.118342:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.118350:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.118356:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.118367:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.118434:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.118443:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f24ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81022976->81027071] (req 81022976->81027071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.118844:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11141us (11711us total) trans 4294987088 rc 0/0 00000100:00100000:2.0:1789760794.118857:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19792 00000100:00100000:2.0:1789760794.118869:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724213392260 time 724232289393 state 0x2 ref 0 00000100:00100000:1.0:1789760794.124046:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.124557:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922304 00000100:00100000:2.0:1789760794.124570:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922304 00000100:00100000:2.0:1789760794.124654:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724213392260 time 724232289393 state 0x0 ref 1 00000100:00100000:2.0:1789760794.124673:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724232549809 time 724232289393 state 0x1 ref 1 00000100:00100000:2.0:1789760794.124699:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724232549809 time 724249305320 state 0x0 ref 1 00000100:00100000:2.0:1789760794.124711:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503922304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.124772:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81027072, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a48 00010000:00010000:2.0:1789760794.124784:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.124806:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.124811:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.124818:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.124835:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.135494:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760794.144918:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987089 is committed 00002000:00010000:2.0:1789760794.145667:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81027072 -> 81031168 00002000:00010000:2.0:1789760794.145674:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158280 -> 158288 00010000:00010000:2.0:1789760794.145693:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.145709:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.145720:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.145728:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.145734:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.145745:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.145781:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.145789:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81027072->81031167] (req 81027072->81031167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.146083:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503922304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 21366us (22045us total) trans 4294987089 rc 0/0 00000100:00100000:2.0:1789760794.146096:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19793 00000100:00100000:2.0:1789760794.146108:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724232549809 time 724249305320 state 0x2 ref 0 00000100:00100000:0.0:1789760794.161639:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.161791:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922432 00000100:00100000:2.0:1789760794.161803:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922432 00000100:00100000:2.0:1789760794.161880:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724232549809 time 724249305320 state 0x0 ref 1 00000100:00100000:2.0:1789760794.161898:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724249565736 time 724249305320 state 0x1 ref 1 00000100:00100000:2.0:1789760794.161957:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724249565736 time 724286563000 state 0x0 ref 1 00000100:00100000:2.0:1789760794.161970:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.162038:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81031168, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a50 00010000:00010000:2.0:1789760794.162049:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.162067:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.162074:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.162081:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.162103:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.170920:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987090 is committed 00002000:00010000:0.0:1789760794.171055:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81031168 -> 81035264 00002000:00010000:0.0:1789760794.171249:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158288 -> 158296 00010000:00010000:0.0:1789760794.171268:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.171283:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.171292:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.171300:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.171305:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.171315:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.171343:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.171351:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f24fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81031168->81035263] (req 81031168->81035263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.172348:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10367us (10709us total) trans 4294987090 rc 0/0 00000100:00100000:0.0:1789760794.172363:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19794 00000100:00100000:0.0:1789760794.172376:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724249565736 time 724286563000 state 0x2 ref 0 00000100:00100000:0.0:1789760794.185855:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.186696:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922560 00000100:00100000:3.0:1789760794.186711:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922560 00000100:00100000:3.0:1789760794.186789:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724249565736 time 724286563000 state 0x0 ref 1 00000100:00100000:3.0:1789760794.186811:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724286823416 time 724286563000 state 0x1 ref 1 00000100:00100000:3.0:1789760794.186852:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724286823416 time 724311458206 state 0x0 ref 1 00000100:00100000:3.0:1789760794.186867:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.186932:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81035264, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a58 00010000:00010000:3.0:1789760794.186944:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.186966:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760794.186972:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.186979:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.186996:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.194134:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987091 is committed 00002000:00010000:3.0:1789760794.194432:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81035264 -> 81039360 00002000:00010000:3.0:1789760794.194439:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158296 -> 158304 00010000:00010000:3.0:1789760794.194455:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.194471:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.194492:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.194500:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.194506:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.194527:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.194558:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.194566:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2501 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81035264->81039359] (req 81035264->81039359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.194706:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7834us (8858us total) trans 4294987091 rc 0/0 00000100:00100000:3.0:1789760794.194718:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19795 00000100:00100000:3.0:1789760794.194731:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724286823416 time 724311458206 state 0x2 ref 0 00000100:00100000:1.0:1789760794.202354:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.202492:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922688 00000100:00100000:0.0:1789760794.202507:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922688 00000100:00100000:0.0:1789760794.202597:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724286823416 time 724311458206 state 0x0 ref 1 00000100:00100000:0.0:1789760794.202616:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724311718622 time 724311458206 state 0x1 ref 1 00000100:00100000:0.0:1789760794.202652:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724311718622 time 724327257730 state 0x0 ref 1 00000100:00100000:0.0:1789760794.202666:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.202735:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81039360, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a60 00010000:00010000:0.0:1789760794.202747:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202768:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.202775:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.202782:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202800:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.211520:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987092 is committed 00002000:00010000:0.0:1789760794.211907:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81039360 -> 81043456 00002000:00010000:0.0:1789760794.211918:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158304 -> 158312 00010000:00010000:0.0:1789760794.211940:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.211988:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.211999:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.212014:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.212020:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.212029:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.212055:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.212064:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2508 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81039360->81043455] (req 81039360->81043455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.212349:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9678us (10003us total) trans 4294987092 rc 0/0 00000100:00100000:0.0:1789760794.212362:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19796 00000100:00100000:0.0:1789760794.212373:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724311718622 time 724327257730 state 0x2 ref 0 00000100:00100000:0.0:1789760794.218583:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.218693:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922816 00000100:00100000:1.0:1789760794.218705:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922816 00000100:00100000:1.0:1789760794.218778:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724311718622 time 724327257730 state 0x0 ref 1 00000100:00100000:1.0:1789760794.218797:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724327518146 time 724327257730 state 0x1 ref 1 00000100:00100000:1.0:1789760794.218831:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724327518146 time 724343437111 state 0x0 ref 1 00000100:00100000:1.0:1789760794.218846:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.218914:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81043456, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a68 00010000:00010000:1.0:1789760794.218925:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.218945:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.218950:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.218958:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.218976:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.228051:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987093 is committed 00002000:00010000:1.0:1789760794.232190:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81043456 -> 81047552 00002000:00010000:1.0:1789760794.232199:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158312 -> 158320 00010000:00010000:1.0:1789760794.232212:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.232412:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.232424:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.232432:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.232436:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.232455:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.232486:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.232494:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f250f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81043456->81047551] (req 81043456->81047551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.232579:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503922816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13732us (14006us total) trans 4294987093 rc 0/0 00000100:00100000:1.0:1789760794.232589:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19797 00000100:00100000:1.0:1789760794.233362:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724327518146 time 724343437111 state 0x2 ref 0 00000100:00100000:2.0:1789760794.236459:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.236629:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503922944 00000100:00100000:1.0:1789760794.236640:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503922944 00000100:00100000:1.0:1789760794.236697:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724327518146 time 724343437111 state 0x0 ref 1 00000100:00100000:1.0:1789760794.236712:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724343697527 time 724343437111 state 0x1 ref 1 00000100:00100000:1.0:1789760794.236742:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724343697527 time 724361348498 state 0x0 ref 1 00000100:00100000:1.0:1789760794.236754:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503922944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.236800:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81047552, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a70 00010000:00010000:1.0:1789760794.236809:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.236825:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.236831:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.236837:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.236853:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.244306:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987094 is committed 00002000:00010000:2.0:1789760794.244466:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81047552 -> 81051648 00002000:00010000:2.0:1789760794.244472:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158320 -> 158328 00010000:00010000:2.0:1789760794.244495:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.244508:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.244526:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.244533:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.244538:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.244547:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.244571:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.244579:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2516 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81047552->81051647] (req 81047552->81051647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.244664:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503922944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7908us (8211us total) trans 4294987094 rc 0/0 00000100:00100000:2.0:1789760794.244674:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19798 00000100:00100000:2.0:1789760794.244683:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724343697527 time 724361348498 state 0x2 ref 0 00000100:00100000:2.0:1789760794.256601:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.256689:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923072 00000100:00100000:3.0:1789760794.256699:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923072 00000100:00100000:3.0:1789760794.256760:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724343697527 time 724361348498 state 0x0 ref 1 00000100:00100000:3.0:1789760794.256775:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724361608914 time 724361348498 state 0x1 ref 1 00000100:00100000:3.0:1789760794.256809:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724361608914 time 724381414742 state 0x0 ref 1 00000100:00100000:3.0:1789760794.256821:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503923072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.256866:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81051648, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a78 00010000:00010000:3.0:1789760794.256875:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.256890:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760794.256895:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.256902:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.256915:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.268596:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987095 is committed 00002000:00010000:3.0:1789760794.268723:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81051648 -> 81055744 00002000:00010000:3.0:1789760794.268729:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158328 -> 158336 00010000:00010000:3.0:1789760794.269618:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.269638:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.269647:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.269654:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.269659:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.269670:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.269689:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.269703:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f251d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81051648->81055743] (req 81051648->81055743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.270057:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503923072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13232us (13458us total) trans 4294987095 rc 0/0 00000100:00100000:3.0:1789760794.270067:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19799 00000100:00100000:3.0:1789760794.270077:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724361608914 time 724381414742 state 0x2 ref 0 00000100:00100000:0.0:1789760794.283358:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.289290:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923200 00000100:00100000:3.0:1789760794.289305:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923200 00000100:00100000:3.0:1789760794.289367:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724361608914 time 724381414742 state 0x0 ref 1 00000100:00100000:3.0:1789760794.289381:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724381675158 time 724381414742 state 0x1 ref 1 00000100:00100000:3.0:1789760794.289431:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724381675158 time 724414037285 state 0x0 ref 1 00000100:00100000:3.0:1789760794.289445:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503923200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.289500:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81055744, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a80 00010000:00010000:3.0:1789760794.289510:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.289535:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760794.289539:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.289546:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.289564:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.298580:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987096 is committed 00002000:00010000:3.0:1789760794.300377:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81055744 -> 81059840 00002000:00010000:3.0:1789760794.300429:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158336 -> 158344 00010000:00010000:3.0:1789760794.300445:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.300458:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.300467:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.300485:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.300490:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.300500:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.300867:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.300878:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2524 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81055744->81059839] (req 81055744->81059839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.301071:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503923200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11622us (18686us total) trans 4294987096 rc 0/0 00000100:00100000:2.0:1789760794.301082:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19800 00000100:00100000:2.0:1789760794.301092:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724381675158 time 724414037285 state 0x2 ref 0 00000100:00100000:2.0:1789760794.307198:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.308322:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923328 00000100:00100000:1.0:1789760794.308335:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923328 00000100:00100000:1.0:1789760794.308922:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724381675158 time 724414037285 state 0x0 ref 1 00000100:00100000:1.0:1789760794.308943:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724414297701 time 724414037285 state 0x1 ref 1 00000100:00100000:1.0:1789760794.308971:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724414297701 time 724433577099 state 0x0 ref 1 00000100:00100000:1.0:1789760794.308987:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503923328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.309039:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81059840, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a88 00010000:00010000:1.0:1789760794.309048:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.309063:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.309068:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.309075:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.309094:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.315904:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987097 is committed 00002000:00010000:1.0:1789760794.320333:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81059840 -> 81063936 00002000:00010000:1.0:1789760794.320342:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158344 -> 158352 00010000:00010000:1.0:1789760794.320355:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.320368:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.320376:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.320383:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.320401:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.320411:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.320440:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.320449:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f252b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81059840->81063935] (req 81059840->81063935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.321010:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503923328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12022us (13815us total) trans 4294987097 rc 0/0 00000100:00100000:1.0:1789760794.321020:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19801 00000100:00100000:1.0:1789760794.321029:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724414297701 time 724433577099 state 0x2 ref 0 00000100:00100000:3.0:1789760794.326169:0:3639:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc696bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523615232:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760794.326245:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760794.326312:0:3637:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760794.326319:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6ab8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523615360:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760794.326342:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760794.326373:0:8432:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523615232 00000100:00100000:3.0:1789760794.326379:0:8432:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523615232 00000100:00100000:2.0:1789760794.326589:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523615360 00000100:00100000:2.0:1789760794.326594:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523615360 00000100:00100000:2.0:1789760794.326637:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:2.0:1789760794.326646:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523615360:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760794.326673:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760794.326717:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3637:x1876699523615360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 71us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.326726:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:0.0:1789760794.326787:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6879880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3638:1876699523615616:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.326835:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760794.326862:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3638:1876699523616128:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.326885:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760794.326906:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3638:1876699523616640:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.326927:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760794.326946:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1789760794.326952:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6878700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523615744:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.326981:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760794.327170:0:8432:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1789760794.327178:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3638:1876699523616256:0@lo:400:kworker.0 00000100:00100000:3.0:1789760794.327179:0:8432:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523615232:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760794.327196:0:8432:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1789760794.327221:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760794.327239:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3638:1876699523616768:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.327261:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760794.327337:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3636:1876699523615872:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.327369:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760794.327413:0:8432:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3639:x1876699523615232:12345-0@lo:13:osp-pre-1-1.0 Request processed in 236us (1167us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760794.327419:0:8432:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789760794.328663:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523615616 00000100:00100000:2.0:1789760794.328672:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523615616 00000100:00100000:2.0:1789760794.328710:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616128 00000100:00100000:2.0:1789760794.328713:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616128 00000100:00100000:2.0:1789760794.328726:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616640 00000100:00100000:2.0:1789760794.328728:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616640 00000100:00100000:2.0:1789760794.328743:0:15090:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616256 00000100:00100000:2.0:1789760794.328745:0:15090:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616256 00000100:00100000:0.0:1789760794.328747:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616768 00000100:00100000:0.0:1789760794.328753:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616768 00000100:00100000:2.0:1789760794.328763:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 995 00000100:00100000:2.0:1789760794.328772:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3638:x1876699523615616:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.328788:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 996 00000100:00100000:0.0:1789760794.328798:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3638:x1876699523616128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.328843:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3638:x1876699523615616:12345-0@lo:400:kworker.0 Request processed in 72us (2007us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.328851:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 995 00000100:00100000:0.0:1789760794.328859:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3638:x1876699523616128:12345-0@lo:400:kworker.0 Request processed in 62us (1971us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.328868:0:15090:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 997 00000100:00100000:2.0:1789760794.328918:0:15090:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3638:x1876699523616640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.328950:0:15090:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3638:x1876699523616640:12345-0@lo:400:kworker.0 Request processed in 75us (2021us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760794.329023:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523616384:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329051:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760794.329076:0:3636:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1789760794.329083:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc6878e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:afc717a1-11ba-4ef0-ab10-103aa692ca98:3636:1876699523615488:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329111:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760794.329131:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523616000:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329151:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760794.329167:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc7eb6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523616512:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329187:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1789760794.329230:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760794.329234:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760794.329236:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760794.329246:0:15162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616384 00000100:00100000:1.0:1789760794.329250:0:15162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616384 00000100:00100000:1.0:1789760794.329279:0:15162:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616512 00000100:00100000:1.0:1789760794.329282:0:15162:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616512 00000100:00100000:1.0:1789760794.329299:0:15162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:1.0:1789760794.329304:0:15162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523616384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329337:0:15162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3636:x1876699523616384:12345-0@lo:400:kworker.0 Request processed in 33us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760794.329344:0:15162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:1.0:1789760794.329357:0:15162:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:1.0:1789760794.329363:0:15162:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523616512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329385:0:15162:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523616512:12345-0@lo:400:kworker.0 Request processed in 23us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760794.329409:0:15162:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:1.0:1789760794.329426:0:6505:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523615488 00000100:00100000:1.0:1789760794.329430:0:6505:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523615488 00000100:00100000:1.0:1789760794.329452:0:6505:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:1.0:1789760794.329458:0:6505:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3636:x1876699523615488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329500:0:6505:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:afc717a1-11ba-4ef0-ab10-103aa692ca98+24:3636:x1876699523615488:12345-0@lo:400:kworker.0 Request processed in 41us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760794.329507:0:6505:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:1.0:1789760794.329533:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523615744 00000100:00100000:1.0:1789760794.329537:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523615744 00000100:00100000:1.0:1789760794.329558:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616000 00000100:00100000:1.0:1789760794.329561:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616000 00000100:00100000:1.0:1789760794.329573:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:1.0:1789760794.329578:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523615744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329605:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 996 00000100:00100000:1.0:1789760794.329613:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523615744:12345-0@lo:400:kworker.0 Request processed in 33us (2630us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760794.329622:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:0.0:1789760794.329624:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523615872 00000100:00100000:0.0:1789760794.329630:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523615872 00000100:00100000:1.0:1789760794.329639:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:1.0:1789760794.329649:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523616000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329656:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 998 00000100:00100000:0.0:1789760794.329668:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3638:x1876699523616256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329690:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523616000:12345-0@lo:400:kworker.0 Request processed in 43us (536us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760794.329698:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1789760794.329720:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc673aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3638:x1876699523616256:12345-0@lo:400:kworker.0 Request processed in 54us (2499us total) trans 0 rc 0/0 00000100:00080000:1.0:1789760794.329728:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760794.329729:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 998 00000100:00100000:0.0:1789760794.329749:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 999 00000100:00100000:0.0:1789760794.329756:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3638:x1876699523616768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329759:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6878e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:afc717a1-11ba-4ef0-ab10-103aa692ca98:3636:1876699523615488:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329777:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523616000:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329791:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523616512:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329797:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6739f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3638:x1876699523616768:12345-0@lo:400:kworker.0 Request processed in 41us (2534us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760794.329804:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3636:1876699523616384:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329806:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 999 00000100:00100000:0.0:1789760794.329837:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1000 00000100:00100000:0.0:1789760794.329846:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3636:x1876699523615872:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329883:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3636:x1876699523615872:12345-0@lo:400:kworker.0 Request processed in 39us (2514us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760794.329890:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1000 00000100:00100000:1.0:1789760794.329926:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6ab8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3637:1876699523615360:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760794.329931:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6878700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523615744:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329946:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3638:1876699523616256:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329959:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3638:1876699523616768:0@lo:400:kworker.0 00000100:00100000:1.0:1789760794.329971:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3636:1876699523615872:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329975:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc6879880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3638:1876699523615616:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329989:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3638:1876699523616128:0@lo:400:kworker.0 00000100:00100000:0.0:1789760794.329998:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc7eb4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3638:1876699523616640:0@lo:400:kworker.0 00000100:00100000:2.0:1789760794.330273:0:15090:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 997 00000100:00100000:3.0:1789760794.330309:0:3639:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc696bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3639:1876699523615232:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760794.330383:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.330602:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923456 00000100:00100000:2.0:1789760794.330608:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923456 00000100:00100000:2.0:1789760794.330644:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724414297701 time 724433577099 state 0x0 ref 1 00000100:00100000:2.0:1789760794.330657:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724433837515 time 724433577099 state 0x1 ref 1 00000100:00100000:2.0:1789760794.330676:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724433837515 time 724455283020 state 0x0 ref 1 00000100:00100000:2.0:1789760794.330686:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503923456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.330726:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81063936, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a90 00010000:00010000:2.0:1789760794.330735:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.330751:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.330755:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.330761:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.330776:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.340079:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987098 is committed 00002000:00010000:0.0:1789760794.343787:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81063936 -> 81068032 00002000:00010000:0.0:1789760794.343798:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158352 -> 158360 00010000:00010000:0.0:1789760794.343814:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.343828:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.343837:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.343845:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.343849:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.343859:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.343879:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.343886:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2532 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81063936->81068031] (req 81063936->81068031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.344033:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503923456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13343us (13647us total) trans 4294987098 rc 0/0 00000100:00100000:0.0:1789760794.344044:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19802 00000100:00100000:0.0:1789760794.344054:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724433837515 time 724455283020 state 0x2 ref 0 00000100:00100000:0.0:1789760794.347461:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.347736:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923584 00000100:00100000:3.0:1789760794.347747:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923584 00000100:00100000:3.0:1789760794.347798:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724433837515 time 724455283020 state 0x0 ref 1 00000100:00100000:3.0:1789760794.347813:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724455543436 time 724455283020 state 0x1 ref 1 00000100:00100000:3.0:1789760794.347844:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724455543436 time 724472450792 state 0x0 ref 1 00000100:00100000:3.0:1789760794.347856:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503923584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.347907:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81068032, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26a98 00010000:00010000:3.0:1789760794.347916:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.347941:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760794.347946:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.347953:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.347967:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.354963:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987099 is committed 00002000:00010000:3.0:1789760794.355075:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81068032 -> 81072128 00002000:00010000:3.0:1789760794.355081:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158360 -> 158368 00010000:00010000:3.0:1789760794.355095:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.355107:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.355115:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.355124:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.355129:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.355145:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.355165:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.355173:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2539 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81068032->81072127] (req 81068032->81072127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.355286:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503923584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7426us (7830us total) trans 4294987099 rc 0/0 00000100:00100000:3.0:1789760794.355296:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19803 00000100:00100000:3.0:1789760794.355306:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724455543436 time 724472450792 state 0x2 ref 0 00000100:00100000:2.0:1789760794.362824:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.362922:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923712 00000100:00100000:0.0:1789760794.362933:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923712 00000100:00100000:0.0:1789760794.362991:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724455543436 time 724472450792 state 0x0 ref 1 00000100:00100000:0.0:1789760794.363008:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724472711208 time 724472450792 state 0x1 ref 1 00000100:00100000:0.0:1789760794.363041:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724472711208 time 724487647876 state 0x0 ref 1 00000100:00100000:0.0:1789760794.363054:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503923712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.363103:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81072128, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26aa0 00010000:00010000:0.0:1789760794.363112:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.363126:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.363131:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.363138:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.363152:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.367365:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987100 is committed 00002000:00010000:0.0:1789760794.367553:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81072128 -> 81076224 00002000:00010000:0.0:1789760794.367558:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158368 -> 158376 00010000:00010000:0.0:1789760794.367571:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.367584:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.367594:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.367603:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.367608:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.367617:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.367638:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.367647:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2540 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81072128->81076223] (req 81072128->81076223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.367791:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503923712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4733us (4970us total) trans 4294987100 rc 0/0 00000100:00100000:0.0:1789760794.367802:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19804 00000100:00100000:0.0:1789760794.367812:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724472711208 time 724487647876 state 0x2 ref 0 00000100:00100000:1.0:1789760794.378347:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.378536:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923840 00000100:00100000:2.0:1789760794.378545:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923840 00000100:00100000:2.0:1789760794.378594:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724472711208 time 724487647876 state 0x0 ref 1 00000100:00100000:2.0:1789760794.378608:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724487908292 time 724487647876 state 0x1 ref 1 00000100:00100000:2.0:1789760794.378630:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724487908292 time 724503237000 state 0x0 ref 1 00000100:00100000:2.0:1789760794.378640:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503923840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.378683:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81076224, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26aa8 00010000:00010000:2.0:1789760794.378691:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.378710:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.378714:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.378720:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.378733:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.383167:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987101 is committed 00002000:00010000:2.0:1789760794.383517:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81076224 -> 81080320 00002000:00010000:2.0:1789760794.383523:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158376 -> 158384 00010000:00010000:2.0:1789760794.383536:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.383549:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.383558:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.383566:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.383572:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.383581:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.383608:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.383616:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2547 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81076224->81080319] (req 81076224->81080319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.383750:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503923840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5105us (5406us total) trans 4294987101 rc 0/0 00000100:00100000:2.0:1789760794.383762:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19805 00000100:00100000:2.0:1789760794.383772:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724487908292 time 724503237000 state 0x2 ref 0 00000100:00100000:2.0:1789760794.393550:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.393642:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503923968 00000100:00100000:0.0:1789760794.393652:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503923968 00000100:00100000:0.0:1789760794.393707:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724487908292 time 724503237000 state 0x0 ref 1 00000100:00100000:0.0:1789760794.393722:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724503497416 time 724503237000 state 0x1 ref 1 00000100:00100000:0.0:1789760794.393756:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724503497416 time 724518362172 state 0x0 ref 1 00000100:00100000:0.0:1789760794.393770:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503923968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.393817:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81080320, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ab0 00010000:00010000:0.0:1789760794.393827:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.393843:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.393848:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.393854:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.393868:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.397744:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987102 is committed 00002000:00010000:0.0:1789760794.407327:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81080320 -> 81084416 00002000:00010000:0.0:1789760794.407337:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158384 -> 158392 00010000:00010000:0.0:1789760794.407351:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407365:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407375:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407484:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.407490:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407501:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407522:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407531:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f254e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81080320->81084415] (req 81080320->81084415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.407680:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503923968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13908us (14135us total) trans 4294987102 rc 0/0 00000100:00100000:0.0:1789760794.407691:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19806 00000100:00100000:0.0:1789760794.407702:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724503497416 time 724518362172 state 0x2 ref 0 00000100:00100000:0.0:1789760794.414074:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.414656:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924096 00000100:00100000:3.0:1789760794.414666:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924096 00000100:00100000:3.0:1789760794.414722:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724503497416 time 724518362172 state 0x0 ref 1 00000100:00100000:3.0:1789760794.414736:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724518622588 time 724518362172 state 0x1 ref 1 00000100:00100000:3.0:1789760794.414820:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724518622588 time 724539426317 state 0x0 ref 1 00000100:00100000:3.0:1789760794.414833:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503924096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.414878:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81084416, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ab8 00010000:00010000:3.0:1789760794.414886:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.414901:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760794.414905:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.414911:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.414926:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.424005:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987103 is committed 00002000:00010000:3.0:1789760794.424134:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81084416 -> 81088512 00002000:00010000:3.0:1789760794.424139:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158392 -> 158400 00010000:00010000:3.0:1789760794.424153:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.424166:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.424176:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.424183:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.424188:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.424198:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.424223:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.424232:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2555 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81084416->81088511] (req 81084416->81088511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.424486:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503924096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9559us (10327us total) trans 4294987103 rc 0/0 00000100:00100000:3.0:1789760794.424497:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19807 00000100:00100000:3.0:1789760794.424506:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724518622588 time 724539426317 state 0x2 ref 0 00000100:00100000:1.0:1789760794.430449:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.432041:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924224 00000100:00100000:2.0:1789760794.432055:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924224 00000100:00100000:2.0:1789760794.432119:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724518622588 time 724539426317 state 0x0 ref 1 00000100:00100000:2.0:1789760794.432133:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724539686733 time 724539426317 state 0x1 ref 1 00000100:00100000:2.0:1789760794.432220:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724539686733 time 724556825885 state 0x0 ref 1 00000100:00100000:2.0:1789760794.432234:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.432283:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81088512, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ac0 00010000:00010000:2.0:1789760794.432292:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.432315:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.432319:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.432326:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.432341:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.440166:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987104 is committed 00002000:00010000:2.0:1789760794.440526:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81088512 -> 81092608 00002000:00010000:2.0:1789760794.440530:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158400 -> 158408 00010000:00010000:2.0:1789760794.440543:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.440554:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.440562:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.440569:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.440573:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.440582:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.440606:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.440614:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f255c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81088512->81092607] (req 81088512->81092607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.440706:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8470us (10262us total) trans 4294987104 rc 0/0 00000100:00100000:2.0:1789760794.440717:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19808 00000100:00100000:2.0:1789760794.440728:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724539686733 time 724556825885 state 0x2 ref 0 00000100:00100000:2.0:1789760794.451759:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.451939:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924352 00000100:00100000:1.0:1789760794.451950:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924352 00000100:00100000:1.0:1789760794.452007:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724539686733 time 724556825885 state 0x0 ref 1 00000100:00100000:1.0:1789760794.452022:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724557086301 time 724556825885 state 0x1 ref 1 00000100:00100000:1.0:1789760794.452058:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724557086301 time 724576665136 state 0x0 ref 1 00000100:00100000:1.0:1789760794.452071:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503924352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.452116:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81092608, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ac8 00010000:00010000:1.0:1789760794.452125:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.452153:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.452158:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.452164:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.452178:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.456928:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987105 is committed 00002000:00010000:1.0:1789760794.457035:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81092608 -> 81096704 00002000:00010000:1.0:1789760794.457040:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158408 -> 158416 00010000:00010000:1.0:1789760794.457053:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457065:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457075:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457089:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.457094:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457104:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457122:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.457131:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2563 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81092608->81096703] (req 81092608->81096703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.457279:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503924352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5205us (5525us total) trans 4294987105 rc 0/0 00000100:00100000:1.0:1789760794.457290:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19809 00000100:00100000:1.0:1789760794.457301:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724557086301 time 724576665136 state 0x2 ref 0 00000100:00100000:0.0:1789760794.466806:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.466956:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924480 00000100:00100000:2.0:1789760794.466966:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924480 00000100:00100000:2.0:1789760794.467021:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724557086301 time 724576665136 state 0x0 ref 1 00000100:00100000:2.0:1789760794.467036:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724576925552 time 724576665136 state 0x1 ref 1 00000100:00100000:2.0:1789760794.467068:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724576925552 time 724591674462 state 0x0 ref 1 00000100:00100000:2.0:1789760794.467082:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503924480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.467126:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81096704, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ad0 00010000:00010000:2.0:1789760794.467136:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.467157:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.467162:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.467169:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.467183:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.471627:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987106 is committed 00002000:00010000:2.0:1789760794.471734:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81096704 -> 81100800 00002000:00010000:2.0:1789760794.471739:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158416 -> 158424 00010000:00010000:2.0:1789760794.471753:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471765:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471775:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471783:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.471788:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471796:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471822:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471831:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f256a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81096704->81100799] (req 81096704->81100799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.471934:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503924480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4840us (5122us total) trans 4294987106 rc 0/0 00000100:00100000:2.0:1789760794.471945:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19810 00000100:00100000:2.0:1789760794.471954:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724576925552 time 724591674462 state 0x2 ref 0 00000100:00100000:3.0:1789760794.475248:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.477704:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924608 00000100:00100000:1.0:1789760794.477719:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924608 00000100:00100000:1.0:1789760794.477788:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724576925552 time 724591674462 state 0x0 ref 1 00000100:00100000:1.0:1789760794.477804:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724591934878 time 724591674462 state 0x1 ref 1 00000100:00100000:1.0:1789760794.477838:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724591934878 time 724602443946 state 0x0 ref 1 00000100:00100000:1.0:1789760794.477852:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503924608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.477899:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81100800, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ad8 00010000:00010000:1.0:1789760794.477908:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.477939:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.477945:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.477952:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.477966:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.484364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987107 is committed 00002000:00010000:1.0:1789760794.484528:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81100800 -> 81104896 00002000:00010000:1.0:1789760794.484533:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158424 -> 158432 00010000:00010000:1.0:1789760794.484547:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.484559:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.484567:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.484574:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.484579:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.484587:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.484608:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.484616:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2571 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81100800->81104895] (req 81100800->81104895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.484716:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503924608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6862us (9472us total) trans 4294987107 rc 0/0 00000100:00100000:1.0:1789760794.484726:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19811 00000100:00100000:1.0:1789760794.484736:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724591934878 time 724602443946 state 0x2 ref 0 00000100:00100000:0.0:1789760794.492606:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.492758:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924736 00000100:00100000:2.0:1789760794.492769:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924736 00000100:00100000:2.0:1789760794.492830:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724591934878 time 724602443946 state 0x0 ref 1 00000100:00100000:2.0:1789760794.492848:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724602704362 time 724602443946 state 0x1 ref 1 00000100:00100000:2.0:1789760794.492882:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724602704362 time 724617487846 state 0x0 ref 1 00000100:00100000:2.0:1789760794.492894:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.492942:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81104896, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ae0 00010000:00010000:2.0:1789760794.492952:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.492974:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.492979:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.492986:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.493001:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.500351:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987108 is committed 00002000:00010000:0.0:1789760794.506868:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81104896 -> 81108992 00002000:00010000:0.0:1789760794.506878:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158432 -> 158440 00010000:00010000:0.0:1789760794.506892:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.506909:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.506919:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.506926:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.506931:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.506941:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.506960:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.506968:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2578 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81104896->81108991] (req 81104896->81108991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.507076:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14178us (14474us total) trans 4294987108 rc 0/0 00000100:00100000:0.0:1789760794.507087:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19812 00000100:00100000:0.0:1789760794.507097:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724602704362 time 724617487846 state 0x2 ref 0 00000100:00100000:0.0:1789760794.515675:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.515771:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924864 00000100:00100000:2.0:1789760794.515781:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924864 00000100:00100000:2.0:1789760794.515835:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724602704362 time 724617487846 state 0x0 ref 1 00000100:00100000:2.0:1789760794.515849:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724617748262 time 724617487846 state 0x1 ref 1 00000100:00100000:2.0:1789760794.515879:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724617748262 time 724640486095 state 0x0 ref 1 00000100:00100000:2.0:1789760794.515890:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.515935:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81108992, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26ae8 00010000:00010000:2.0:1789760794.515943:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.515959:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.515963:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.515970:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.515984:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.520759:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987109 is committed 00002000:00010000:2.0:1789760794.521539:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81108992 -> 81113088 00002000:00010000:2.0:1789760794.521546:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158440 -> 158448 00010000:00010000:2.0:1789760794.521559:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.521571:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.521581:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.521589:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.521594:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.521605:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.521626:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.521635:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f257f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81108992->81113087] (req 81108992->81113087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.521783:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5889us (6112us total) trans 4294987109 rc 0/0 00000100:00100000:2.0:1789760794.521795:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19813 00000100:00100000:2.0:1789760794.521806:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724617748262 time 724640486095 state 0x2 ref 0 00000100:00100000:3.0:1789760794.528023:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.531439:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503924992 00000100:00100000:0.0:1789760794.531452:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503924992 00000100:00100000:2.0:1789760794.531761:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724617748262 time 724640486095 state 0x0 ref 1 00000100:00100000:2.0:1789760794.531779:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724640746511 time 724640486095 state 0x1 ref 1 00000100:00100000:2.0:1789760794.531813:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724640746511 time 724656419267 state 0x0 ref 1 00000100:00100000:2.0:1789760794.531828:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.531875:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81113088, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26af0 00010000:00010000:2.0:1789760794.531885:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.531902:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.531907:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.531913:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.531928:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.540584:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987110 is committed 00002000:00010000:2.0:1789760794.540744:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81113088 -> 81117184 00002000:00010000:2.0:1789760794.540749:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158448 -> 158456 00010000:00010000:2.0:1789760794.540762:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.540775:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.540785:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.540792:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.540797:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.540805:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.540824:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.540832:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2586 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81113088->81117183] (req 81113088->81117183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.540928:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503924992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9099us (12910us total) trans 4294987110 rc 0/0 00000100:00100000:2.0:1789760794.540940:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19814 00000100:00100000:2.0:1789760794.540950:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724640746511 time 724656419267 state 0x2 ref 0 00000100:00100000:0.0:1789760794.553089:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.557727:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503925120 00000100:00100000:1.0:1789760794.557743:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503925120 00000100:00100000:1.0:1789760794.558027:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724640746511 time 724656419267 state 0x0 ref 1 00000100:00100000:1.0:1789760794.558043:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724656679683 time 724656419267 state 0x1 ref 1 00000100:00100000:1.0:1789760794.558148:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724656679683 time 724682753455 state 0x0 ref 1 00000100:00100000:1.0:1789760794.558163:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.558211:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81117184, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26af8 00010000:00010000:1.0:1789760794.558220:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558235:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.558239:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.558246:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558259:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.569365:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987111 is committed 00002000:00010000:1.0:1789760794.569999:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81117184 -> 81121280 00002000:00010000:1.0:1789760794.570004:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158456 -> 158464 00010000:00010000:1.0:1789760794.570018:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.570030:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.570039:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.570047:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.570051:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.570065:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.570085:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.570092:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f258d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81117184->81121279] (req 81117184->81121279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.570184:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12020us (17100us total) trans 4294987111 rc 0/0 00000100:00100000:1.0:1789760794.570194:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19815 00000100:00100000:0.0:1789760794.573829:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.574031:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503925248 00000100:00100000:2.0:1789760794.574041:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503925248 00000100:00100000:2.0:1789760794.578308:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724683013871 time 724682753455 state 0x1 ref 1 00000100:00100000:2.0:1789760794.578435:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724683013871 time 724703040896 state 0x0 ref 1 00000100:00100000:2.0:1789760794.578459:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503925248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.578536:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81121280, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b00 00010000:00010000:2.0:1789760794.578547:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.578563:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.578568:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.578574:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.578588:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.586443:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987112 is committed 00002000:00010000:2.0:1789760794.586658:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81121280 -> 81125376 00002000:00010000:2.0:1789760794.586663:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158464 -> 158472 00010000:00010000:2.0:1789760794.586677:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.586690:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.586706:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.586724:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.586729:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.586739:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.586828:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.586838:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2594 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81121280->81125375] (req 81121280->81125375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.586980:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503925248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8528us (13154us total) trans 4294987112 rc 0/0 00000100:00100000:2.0:1789760794.586991:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19816 00000100:00100000:2.0:1789760794.587001:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724683013871 time 724703040896 state 0x2 ref 0 00000100:00100000:3.0:1789760794.602454:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.602880:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503925376 00000100:00100000:2.0:1789760794.602890:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503925376 00000100:00100000:2.0:1789760794.602958:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724683013871 time 724703040896 state 0x0 ref 1 00000100:00100000:2.0:1789760794.602975:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724703301312 time 724703040896 state 0x1 ref 1 00000100:00100000:2.0:1789760794.603005:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724703301312 time 724727611636 state 0x0 ref 1 00000100:00100000:2.0:1789760794.603018:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503925376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.603062:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81125376, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b08 00010000:00010000:2.0:1789760794.603072:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.603087:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.603092:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.603098:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.603111:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.615840:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987113 is committed 00002000:00010000:1.0:1789760794.616239:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81125376 -> 81129472 00002000:00010000:1.0:1789760794.616245:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158472 -> 158480 00010000:00010000:1.0:1789760794.616259:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616272:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616282:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616290:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.616296:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616306:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616325:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616333:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f259b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81125376->81129471] (req 81125376->81129471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.616501:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503925376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13481us (14051us total) trans 4294987113 rc 0/0 00000100:00100000:1.0:1789760794.616519:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19817 00000100:00100000:1.0:1789760794.616530:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724703301312 time 724727611636 state 0x2 ref 0 00000100:00100000:0.0:1789760794.626046:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.626145:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503925504 00000100:00100000:2.0:1789760794.626156:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503925504 00000100:00100000:2.0:1789760794.626218:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724703301312 time 724727611636 state 0x0 ref 1 00000100:00100000:2.0:1789760794.626232:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724727872052 time 724727611636 state 0x1 ref 1 00000100:00100000:2.0:1789760794.626260:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724727872052 time 724750866656 state 0x0 ref 1 00000100:00100000:2.0:1789760794.626272:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.626322:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81129472, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b10 00010000:00010000:2.0:1789760794.626331:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.626346:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.626351:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.626356:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.626369:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.637180:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987114 is committed 00002000:00010000:2.0:1789760794.637348:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81129472 -> 81133568 00002000:00010000:2.0:1789760794.637353:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158480 -> 158488 00010000:00010000:2.0:1789760794.637365:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.637377:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.637385:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.637426:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.637431:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.637446:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.637465:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.637483:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f25a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81129472->81133567] (req 81129472->81133567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.637581:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11306us (11540us total) trans 4294987114 rc 0/0 00000100:00100000:2.0:1789760794.637597:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19818 00000100:00100000:2.0:1789760794.637607:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724727872052 time 724750866656 state 0x2 ref 0 00000100:00100000:2.0:1789760794.643168:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.647143:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503925632 00000100:00100000:1.0:1789760794.647158:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503925632 00000100:00100000:1.0:1789760794.647360:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724727872052 time 724750866656 state 0x0 ref 1 00000100:00100000:1.0:1789760794.647375:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724751127072 time 724750866656 state 0x1 ref 1 00000100:00100000:1.0:1789760794.647780:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724751127072 time 724772384814 state 0x0 ref 1 00000100:00100000:1.0:1789760794.647796:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.647851:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81133568, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b18 00010000:00010000:1.0:1789760794.647862:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647878:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.647883:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.647890:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647903:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.654337:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987115 is committed 00002000:00010000:0.0:1789760794.654722:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81133568 -> 81137664 00002000:00010000:0.0:1789760794.654727:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158488 -> 158496 00010000:00010000:0.0:1789760794.654740:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.654752:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.654761:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.654768:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.654772:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.654781:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.654799:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.654807:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f25a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81133568->81137663] (req 81133568->81137663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.654938:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7141us (11775us total) trans 4294987115 rc 0/0 00000100:00100000:0.0:1789760794.654948:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19819 00000100:00100000:0.0:1789760794.654958:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724751127072 time 724772384814 state 0x2 ref 0 00000100:00100000:1.0:1789760794.658148:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.658599:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503925760 00000100:00100000:0.0:1789760794.658609:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503925760 00000100:00100000:0.0:1789760794.658661:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724751127072 time 724772384814 state 0x0 ref 1 00000100:00100000:0.0:1789760794.658675:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724772645230 time 724772384814 state 0x1 ref 1 00000100:00100000:0.0:1789760794.658748:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724772645230 time 724783353105 state 0x0 ref 1 00000100:00100000:0.0:1789760794.658761:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503925760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.658808:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81137664, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b20 00010000:00010000:0.0:1789760794.658817:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.658832:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.658837:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.658844:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.658857:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.663807:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987116 is committed 00002000:00010000:0.0:1789760794.664008:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81137664 -> 81141760 00002000:00010000:0.0:1789760794.664013:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158496 -> 158504 00010000:00010000:0.0:1789760794.664026:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.664039:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.664048:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.664055:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.664061:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.664069:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.664089:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.664098:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f25b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81137664->81141759] (req 81137664->81141759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.664371:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503925760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5607us (6226us total) trans 4294987116 rc 0/0 00000100:00100000:0.0:1789760794.664384:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19820 00000100:00100000:0.0:1789760794.665660:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724772645230 time 724783353105 state 0x2 ref 0 00000100:00100000:2.0:1789760794.668752:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.668864:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503925888 00000100:00100000:1.0:1789760794.668874:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503925888 00000100:00100000:1.0:1789760794.668930:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724772645230 time 724783353105 state 0x0 ref 1 00000100:00100000:1.0:1789760794.668944:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724783613521 time 724783353105 state 0x1 ref 1 00000100:00100000:1.0:1789760794.669005:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724783613521 time 724793611159 state 0x0 ref 1 00000100:00100000:1.0:1789760794.669018:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.669068:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81141760, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b28 00010000:00010000:1.0:1789760794.669078:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.669094:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.669099:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.669107:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.669122:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.674944:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987117 is committed 00002000:00010000:1.0:1789760794.675663:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81141760 -> 81145856 00002000:00010000:1.0:1789760794.675669:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158504 -> 158512 00010000:00010000:1.0:1789760794.675683:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.675696:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.675705:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.675712:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.675718:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.675727:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.675747:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.675760:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81141760->81145855] (req 81141760->81145855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.677754:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503925888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8727us (9001us total) trans 4294987117 rc 0/0 00000100:00100000:0.0:1789760794.677768:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19821 00000100:00100000:0.0:1789760794.677778:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724783613521 time 724793611159 state 0x2 ref 0 00000100:00100000:2.0:1789760794.688092:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.692633:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926016 00000100:00100000:0.0:1789760794.692647:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926016 00000100:00100000:0.0:1789760794.692763:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724783613521 time 724793611159 state 0x0 ref 1 00000100:00100000:0.0:1789760794.692779:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724793871575 time 724793611159 state 0x1 ref 1 00000100:00100000:0.0:1789760794.692867:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724793871575 time 724817472576 state 0x0 ref 1 00000100:00100000:0.0:1789760794.692882:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.692926:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81145856, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b30 00010000:00010000:0.0:1789760794.692936:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.692952:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.692956:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.692962:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.692981:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.698249:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987118 is committed 00002000:00010000:0.0:1789760794.698676:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81145856 -> 81149952 00002000:00010000:0.0:1789760794.698682:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158512 -> 158520 00010000:00010000:0.0:1789760794.698696:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.698709:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.698719:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.698728:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.698732:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.698742:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.698762:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.698770:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81145856->81149951] (req 81145856->81149951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.698909:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6024us (10822us total) trans 4294987118 rc 0/0 00000100:00100000:0.0:1789760794.698920:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19822 00000100:00100000:0.0:1789760794.698930:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724793871575 time 724817472576 state 0x2 ref 0 00000100:00100000:1.0:1789760794.708831:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.708928:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926144 00000100:00100000:0.0:1789760794.708938:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926144 00000100:00100000:1.0:1789760794.709166:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724793871575 time 724817472576 state 0x0 ref 1 00000100:00100000:1.0:1789760794.709182:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724817732992 time 724817472576 state 0x1 ref 1 00000100:00100000:1.0:1789760794.709233:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724817732992 time 724833839243 state 0x0 ref 1 00000100:00100000:1.0:1789760794.709245:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503926144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.709291:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81149952, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b38 00010000:00010000:1.0:1789760794.709300:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.709316:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.709320:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.709327:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.709342:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.718034:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987119 is committed 00002000:00010000:1.0:1789760794.724620:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81149952 -> 81154048 00002000:00010000:1.0:1789760794.724631:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158520 -> 158528 00010000:00010000:1.0:1789760794.724647:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.724660:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.724669:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.724676:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.724681:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.724689:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.724707:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.724714:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f25c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81149952->81154047] (req 81149952->81154047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.724856:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503926144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15607us (16030us total) trans 4294987119 rc 0/0 00000100:00100000:1.0:1789760794.724866:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19823 00000100:00100000:1.0:1789760794.724876:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724817732992 time 724833839243 state 0x2 ref 0 00000100:00100000:2.0:1789760794.732308:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.733152:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926272 00000100:00100000:0.0:1789760794.733162:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926272 00000100:00100000:0.0:1789760794.733368:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724817732992 time 724833839243 state 0x0 ref 1 00000100:00100000:0.0:1789760794.733536:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724834099659 time 724833839243 state 0x1 ref 1 00000100:00100000:0.0:1789760794.733601:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724834099659 time 724858207462 state 0x0 ref 1 00000100:00100000:0.0:1789760794.733614:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.733659:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81154048, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b40 00010000:00010000:0.0:1789760794.733668:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.733683:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.733687:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.733693:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.733705:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.738929:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987120 is committed 00002000:00010000:0.0:1789760794.739064:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81154048 -> 81158144 00002000:00010000:0.0:1789760794.739069:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158528 -> 158536 00010000:00010000:0.0:1789760794.739082:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.739095:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.739104:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.739111:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.739116:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.739125:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.739145:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.739153:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81154048->81158143] (req 81154048->81158143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.739551:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5933us (7246us total) trans 4294987120 rc 0/0 00000100:00100000:0.0:1789760794.739563:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19824 00000100:00100000:0.0:1789760794.739573:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724834099659 time 724858207462 state 0x2 ref 0 00000100:00100000:2.0:1789760794.749130:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.749724:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926400 00000100:00100000:0.0:1789760794.749735:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926400 00000100:00100000:0.0:1789760794.749785:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724834099659 time 724858207462 state 0x0 ref 1 00000100:00100000:0.0:1789760794.749799:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724858467878 time 724858207462 state 0x1 ref 1 00000100:00100000:0.0:1789760794.749818:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724858467878 time 724874425612 state 0x0 ref 1 00000100:00100000:0.0:1789760794.749828:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503926400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.749870:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81158144, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b48 00010000:00010000:0.0:1789760794.749879:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.749893:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.749898:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.749904:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.749918:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.761182:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987121 is committed 00002000:00010000:0.0:1789760794.762968:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81158144 -> 81162240 00002000:00010000:0.0:1789760794.762977:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158536 -> 158544 00010000:00010000:0.0:1789760794.762991:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.763004:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.763014:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.763023:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.763028:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.763038:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.763058:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.763067:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81158144->81162239] (req 81158144->81162239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.763169:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503926400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13335us (14041us total) trans 4294987121 rc 0/0 00000100:00100000:0.0:1789760794.763179:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19825 00000100:00100000:0.0:1789760794.763190:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724858467878 time 724874425612 state 0x2 ref 0 00000100:00100000:1.0:1789760794.768779:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.768876:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926528 00000100:00100000:2.0:1789760794.768885:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926528 00000100:00100000:2.0:1789760794.768932:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724858467878 time 724874425612 state 0x0 ref 1 00000100:00100000:2.0:1789760794.768947:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724874686028 time 724874425612 state 0x1 ref 1 00000100:00100000:2.0:1789760794.768976:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724874686028 time 724893582208 state 0x0 ref 1 00000100:00100000:2.0:1789760794.768987:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.769027:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81162240, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b50 00010000:00010000:2.0:1789760794.769036:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.769052:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.769056:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.769062:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.769076:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.775834:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987122 is committed 00002000:00010000:2.0:1789760794.779794:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81162240 -> 81166336 00002000:00010000:2.0:1789760794.779803:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158544 -> 158552 00010000:00010000:2.0:1789760794.779817:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.779841:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.779851:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.779859:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.779864:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.779873:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.779893:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.779900:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81162240->81166335] (req 81162240->81166335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.780004:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11013us (11229us total) trans 4294987122 rc 0/0 00000100:00100000:2.0:1789760794.780013:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19826 00000100:00100000:2.0:1789760794.780024:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724874686028 time 724893582208 state 0x2 ref 0 00000100:00100000:2.0:1789760794.785536:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.785654:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926656 00000100:00100000:1.0:1789760794.785663:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926656 00000100:00100000:1.0:1789760794.785716:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724874686028 time 724893582208 state 0x0 ref 1 00000100:00100000:1.0:1789760794.785730:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724893842624 time 724893582208 state 0x1 ref 1 00000100:00100000:1.0:1789760794.785758:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724893842624 time 724910364574 state 0x0 ref 1 00000100:00100000:1.0:1789760794.785770:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.785815:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81166336, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x26b58 00010000:00010000:1.0:1789760794.785824:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.785840:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760794.785845:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.785851:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.785866:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.798970:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987123 is committed 00002000:00010000:1.0:1789760794.799199:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81166336 -> 81170432 00002000:00010000:1.0:1789760794.799205:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:1.0:1789760794.799208:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00002000:00010000:1.0:1789760794.799211:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158552 -> 158560 00010000:00010000:1.0:1789760794.799223:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.799236:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.799245:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.799253:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760794.799257:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.799266:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.799533:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.799541:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f25e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81166336->81170431] (req 81166336->81170431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.799638:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13864us (14106us total) trans 4294987123 rc 0/0 00000100:00100000:1.0:1789760794.799650:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19827 00000100:00100000:1.0:1789760794.799660:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724893842624 time 724910364574 state 0x2 ref 0 00000100:00100000:1.0:1789760794.810761:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.810864:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926784 00000100:00100000:0.0:1789760794.810873:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926784 00000100:00100000:0.0:1789760794.810931:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724893842624 time 724910364574 state 0x0 ref 1 00000100:00100000:0.0:1789760794.810946:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724910624990 time 724910364574 state 0x1 ref 1 00000100:00100000:0.0:1789760794.810978:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724910624990 time 724935585091 state 0x0 ref 1 00000100:00100000:0.0:1789760794.810991:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503926784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.811039:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81170432, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b60 00010000:00010000:0.0:1789760794.811049:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.811065:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.811069:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.811074:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.811088:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.820277:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987124 is committed 00002000:00010000:0.0:1789760794.820419:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81170432 -> 81174528 00002000:00010000:0.0:1789760794.820432:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158560 -> 158568 00010000:00010000:0.0:1789760794.820446:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.820465:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.820482:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.820490:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.820495:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.820691:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.820714:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.820721:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f25e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81170432->81174527] (req 81170432->81174527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.820813:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503926784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9819us (10059us total) trans 4294987124 rc 0/0 00000100:00100000:0.0:1789760794.820823:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19828 00000100:00100000:0.0:1789760794.820833:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724910624990 time 724935585091 state 0x2 ref 0 00000100:00100000:3.0:1789760794.823954:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.824799:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503926912 00000100:00100000:0.0:1789760794.824810:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503926912 00000100:00100000:0.0:1789760794.824863:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724910624990 time 724935585091 state 0x0 ref 1 00000100:00100000:0.0:1789760794.824877:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724935845507 time 724935585091 state 0x1 ref 1 00000100:00100000:0.0:1789760794.824896:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724935845507 time 724949503571 state 0x0 ref 1 00000100:00100000:0.0:1789760794.824907:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.824957:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81174528, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b68 00010000:00010000:0.0:1789760794.824966:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.824980:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760794.824994:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.825000:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.825015:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.835229:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987125 is committed 00002000:00010000:0.0:1789760794.835358:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81174528 -> 81178624 00002000:00010000:0.0:1789760794.835363:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158568 -> 158576 00010000:00010000:0.0:1789760794.835375:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.835387:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.835617:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.835627:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.835633:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.835643:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.835662:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.835670:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f25ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81174528->81178623] (req 81174528->81178623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.835818:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503926912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10905us (11867us total) trans 4294987125 rc 0/0 00000100:00100000:0.0:1789760794.835828:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19829 00000100:00100000:0.0:1789760794.835838:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724935845507 time 724949503571 state 0x2 ref 0 00000100:00100000:1.0:1789760794.858901:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.858998:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927040 00000100:00100000:2.0:1789760794.859009:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927040 00000100:00100000:2.0:1789760794.859064:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724935845507 time 724949503571 state 0x0 ref 1 00000100:00100000:2.0:1789760794.859082:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724949763987 time 724949503571 state 0x1 ref 1 00000100:00100000:2.0:1789760794.859111:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724949763987 time 724983717606 state 0x0 ref 1 00000100:00100000:2.0:1789760794.859123:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503927040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.859168:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81178624, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b70 00010000:00010000:2.0:1789760794.859177:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.859191:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760794.859205:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.859212:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.859227:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.864685:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987126 is committed 00002000:00010000:2.0:1789760794.869317:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81178624 -> 81182720 00002000:00010000:2.0:1789760794.869326:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158576 -> 158584 00010000:00010000:2.0:1789760794.869339:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.869352:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.869361:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.869370:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.869375:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.869383:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.869415:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.870957:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f25f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81178624->81182719] (req 81178624->81182719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.871079:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503927040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11951us (12179us total) trans 4294987126 rc 0/0 00000100:00100000:2.0:1789760794.871089:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19830 00000100:00100000:2.0:1789760794.871098:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724949763987 time 724983717606 state 0x2 ref 0 00000100:00100000:1.0:1789760794.873629:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.873936:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927168 00000100:00100000:3.0:1789760794.873946:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927168 00000100:00100000:3.0:1789760794.874006:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724949763987 time 724983717606 state 0x0 ref 1 00000100:00100000:3.0:1789760794.874020:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724983978022 time 724983717606 state 0x1 ref 1 00000100:00100000:3.0:1789760794.874049:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724983978022 time 724998656534 state 0x0 ref 1 00000100:00100000:3.0:1789760794.874061:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503927168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.874103:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81182720, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b78 00010000:00010000:3.0:1789760794.874112:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874126:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760794.874130:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.874136:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.874148:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.879228:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987127 is committed 00002000:00010000:3.0:1789760794.879330:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81182720 -> 81186816 00002000:00010000:3.0:1789760794.879335:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158584 -> 158592 00010000:00010000:3.0:1789760794.879347:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.879360:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.879369:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.879376:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.879380:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.879437:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.879457:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.879465:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f25fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81182720->81186815] (req 81182720->81186815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.879564:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503927168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5501us (5940us total) trans 4294987127 rc 0/0 00000100:00100000:3.0:1789760794.879573:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19831 00000100:00100000:3.0:1789760794.879582:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724983978022 time 724998656534 state 0x2 ref 0 00000100:00100000:3.0:1789760794.898380:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.898521:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927296 00000100:00100000:0.0:1789760794.898532:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927296 00000100:00100000:0.0:1789760794.898585:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724983978022 time 724998656534 state 0x0 ref 1 00000100:00100000:0.0:1789760794.898600:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724998916950 time 724998656534 state 0x1 ref 1 00000100:00100000:0.0:1789760794.898630:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724998916950 time 725023236344 state 0x0 ref 1 00000100:00100000:0.0:1789760794.898641:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503927296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.898687:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81186816, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b80 00010000:00010000:0.0:1789760794.898696:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.898712:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760794.898717:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.898723:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.898737:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.904665:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987128 is committed 00002000:00010000:0.0:1789760794.905114:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81186816 -> 81190912 00002000:00010000:0.0:1789760794.905120:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158592 -> 158600 00010000:00010000:0.0:1789760794.905133:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.905144:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.905152:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.905163:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.905167:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.905174:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.905203:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.905211:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2604 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81186816->81190911] (req 81186816->81190911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.905296:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503927296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6651us (6921us total) trans 4294987128 rc 0/0 00000100:00100000:0.0:1789760794.905305:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19832 00000100:00100000:0.0:1789760794.905315:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724998916950 time 725023236344 state 0x2 ref 0 00000100:00100000:1.0:1789760794.911721:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.912122:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927424 00000100:00100000:0.0:1789760794.912132:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927424 00000100:00100000:0.0:1789760794.912184:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 724998916950 time 725023236344 state 0x0 ref 1 00000100:00100000:0.0:1789760794.912323:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725023496760 time 725023236344 state 0x1 ref 1 00000100:00100000:0.0:1789760794.912351:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725023496760 time 725036958503 state 0x0 ref 1 00000100:00100000:0.0:1789760794.912367:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.913270:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81190912, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b88 00010000:00010000:0.0:1789760794.913290:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.913307:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760794.913311:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.913317:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.913331:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.922568:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987129 is committed 00002000:00010000:0.0:1789760794.922776:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81190912 -> 81195008 00002000:00010000:0.0:1789760794.922781:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158600 -> 158608 00010000:00010000:0.0:1789760794.922793:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.922805:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.922814:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.922827:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.922832:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.922842:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.922861:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.922868:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f260b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81190912->81195007] (req 81190912->81195007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.922964:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10592us (11573us total) trans 4294987129 rc 0/0 00000100:00100000:0.0:1789760794.922975:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19833 00000100:00100000:0.0:1789760794.922993:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725023496760 time 725036958503 state 0x2 ref 0 00000100:00100000:2.0:1789760794.929176:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.932740:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927552 00000100:00100000:0.0:1789760794.932755:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927552 00000100:00100000:0.0:1789760794.932812:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725023496760 time 725036958503 state 0x0 ref 1 00000100:00100000:0.0:1789760794.932828:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725037218919 time 725036958503 state 0x1 ref 1 00000100:00100000:0.0:1789760794.932861:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725037218919 time 725057467564 state 0x0 ref 1 00000100:00100000:0.0:1789760794.932873:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.932916:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81195008, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b90 00010000:00010000:0.0:1789760794.932925:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.932941:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760794.932945:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.932951:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.932964:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.938800:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987130 is committed 00002000:00010000:0.0:1789760794.938907:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81195008 -> 81199104 00002000:00010000:0.0:1789760794.938912:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158608 -> 158616 00010000:00010000:0.0:1789760794.938925:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.938937:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.938945:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.938952:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.938957:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.938966:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.939154:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.939165:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2612 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81195008->81199103] (req 81195008->81199103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.939329:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6453us (10158us total) trans 4294987130 rc 0/0 00000100:00100000:0.0:1789760794.939339:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19834 00000100:00100000:0.0:1789760794.939348:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725037218919 time 725057467564 state 0x2 ref 0 00000100:00100000:3.0:1789760794.942236:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760794.942506:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927680 00000100:00100000:1.0:1789760794.942525:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927680 00000100:00100000:1.0:1789760794.942580:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725037218919 time 725057467564 state 0x0 ref 1 00000100:00100000:1.0:1789760794.942595:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725057727980 time 725057467564 state 0x1 ref 1 00000100:00100000:1.0:1789760794.942621:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725057727980 time 725067227595 state 0x0 ref 1 00000100:00100000:1.0:1789760794.942633:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760794.942681:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81199104, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26b98 00010000:00010000:1.0:1789760794.942690:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.942805:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760794.942810:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.942817:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.942837:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.950434:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987131 is committed 00002000:00010000:3.0:1789760794.950842:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81199104 -> 81203200 00002000:00010000:3.0:1789760794.950847:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158616 -> 158624 00010000:00010000:3.0:1789760794.950861:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.950874:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.950882:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.950889:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.950893:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.950903:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.950922:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.950931:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2619 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81199104->81203199] (req 81199104->81203199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.951197:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8561us (8965us total) trans 4294987131 rc 0/0 00000100:00100000:3.0:1789760794.951208:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19835 00000100:00100000:3.0:1789760794.951217:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725057727980 time 725067227595 state 0x2 ref 0 00000100:00100000:2.0:1789760794.954526:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.954626:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927808 00000100:00100000:0.0:1789760794.954636:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927808 00000100:00100000:0.0:1789760794.954685:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725057727980 time 725067227595 state 0x0 ref 1 00000100:00100000:0.0:1789760794.954820:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725067488011 time 725067227595 state 0x1 ref 1 00000100:00100000:0.0:1789760794.954882:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725067488011 time 725079486978 state 0x0 ref 1 00000100:00100000:0.0:1789760794.954896:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503927808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.954945:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81203200, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26ba0 00010000:00010000:0.0:1789760794.954955:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.954971:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760794.954977:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.954983:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.954997:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.961110:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987132 is committed 00002000:00010000:0.0:1789760794.961229:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81203200 -> 81207296 00002000:00010000:0.0:1789760794.961234:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158624 -> 158632 00010000:00010000:0.0:1789760794.961247:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.961259:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.961267:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.961274:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.961279:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.961287:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.961307:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.961321:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2620 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81203200->81207295] (req 81203200->81207295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.961619:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503927808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6721us (7097us total) trans 4294987132 rc 0/0 00000100:00100000:0.0:1789760794.961629:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19836 00000100:00100000:0.0:1789760794.961639:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725067488011 time 725079486978 state 0x2 ref 0 00000100:00100000:2.0:1789760794.965984:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760794.966072:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503927936 00000100:00100000:0.0:1789760794.966082:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503927936 00000100:00100000:0.0:1789760794.966134:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725067488011 time 725079486978 state 0x0 ref 1 00000100:00100000:0.0:1789760794.966149:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725079747394 time 725079486978 state 0x1 ref 1 00000100:00100000:0.0:1789760794.966180:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725079747394 time 725090786293 state 0x0 ref 1 00000100:00100000:0.0:1789760794.966192:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760794.966236:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81207296, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26ba8 00010000:00010000:0.0:1789760794.966245:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.966262:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760794.966266:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.966272:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.966291:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.970354:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987133 is committed 00002000:00010000:0.0:1789760794.970540:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81207296 -> 81211392 00002000:00010000:0.0:1789760794.970545:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158632 -> 158640 00010000:00010000:0.0:1789760794.970557:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.970569:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.970578:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.970586:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760794.970591:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.970601:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.970621:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.970629:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2627 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81207296->81211391] (req 81207296->81211391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.970717:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503927936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4523us (4738us total) trans 4294987133 rc 0/0 00000100:00100000:0.0:1789760794.970727:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19837 00000100:00100000:0.0:1789760794.970737:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725079747394 time 725090786293 state 0x2 ref 0 00000100:00100000:3.0:1789760794.974149:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.975155:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928064 00000100:00100000:2.0:1789760794.975166:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928064 00000100:00100000:2.0:1789760794.975223:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725079747394 time 725090786293 state 0x0 ref 1 00000100:00100000:2.0:1789760794.975237:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725091046709 time 725090786293 state 0x1 ref 1 00000100:00100000:2.0:1789760794.975324:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725091046709 time 725099929441 state 0x0 ref 1 00000100:00100000:2.0:1789760794.975338:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503928064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.975415:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81211392, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bb0 00010000:00010000:2.0:1789760794.975425:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.975441:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760794.975445:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.975452:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.975465:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.980318:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987134 is committed 00002000:00010000:2.0:1789760794.980688:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81211392 -> 81215488 00002000:00010000:2.0:1789760794.980694:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158640 -> 158648 00010000:00010000:2.0:1789760794.980706:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980720:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980734:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980741:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760794.980746:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980755:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980774:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.980782:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f262e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81211392->81215487] (req 81211392->81215487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.980910:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503928064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5569us (6764us total) trans 4294987134 rc 0/0 00000100:00100000:2.0:1789760794.980920:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19838 00000100:00100000:2.0:1789760794.980930:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725091046709 time 725099929441 state 0x2 ref 0 00000100:00100000:2.0:1789760794.984710:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760794.984785:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928192 00000100:00100000:3.0:1789760794.984795:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928192 00000100:00100000:3.0:1789760794.984849:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725091046709 time 725099929441 state 0x0 ref 1 00000100:00100000:3.0:1789760794.984863:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725100189857 time 725099929441 state 0x1 ref 1 00000100:00100000:3.0:1789760794.984894:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725100189857 time 725109501040 state 0x0 ref 1 00000100:00100000:3.0:1789760794.984913:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503928192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760794.984960:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81215488, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bb8 00010000:00010000:3.0:1789760794.984968:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.985096:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760794.985102:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.985108:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.985124:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.992721:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987135 is committed 00002000:00010000:3.0:1789760794.992894:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81215488 -> 81219584 00002000:00010000:3.0:1789760794.992963:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158648 -> 158656 00010000:00010000:3.0:1789760794.992977:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.992989:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.992998:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.993006:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760794.993012:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.993022:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.993042:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.993050:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2635 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81215488->81219583] (req 81215488->81219583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.993464:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503928192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8229us (8435us total) trans 4294987135 rc 0/0 00000100:00100000:3.0:1789760794.993485:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19839 00000100:00100000:3.0:1789760794.993495:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725100189857 time 725109501040 state 0x2 ref 0 00000100:00100000:0.0:1789760794.996801:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760794.997089:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928320 00000100:00100000:2.0:1789760794.997100:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928320 00000100:00100000:2.0:1789760794.997152:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725100189857 time 725109501040 state 0x0 ref 1 00000100:00100000:2.0:1789760794.997171:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725109761456 time 725109501040 state 0x1 ref 1 00000100:00100000:2.0:1789760794.997201:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725109761456 time 725121808035 state 0x0 ref 1 00000100:00100000:2.0:1789760794.997213:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760794.997267:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81219584, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bc0 00010000:00010000:2.0:1789760794.997276:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.997617:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760794.997623:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.997630:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.997647:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.002444:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987136 is committed 00002000:00010000:2.0:1789760795.002719:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81219584 -> 81223680 00002000:00010000:2.0:1789760795.002724:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158656 -> 158664 00010000:00010000:2.0:1789760795.002736:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.002749:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.002758:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.002775:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.002780:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.002789:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.002808:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.002815:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f263c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81219584->81223679] (req 81219584->81223679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.003056:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5839us (6256us total) trans 4294987136 rc 0/0 00000100:00100000:2.0:1789760795.003067:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19840 00000100:00100000:2.0:1789760795.003076:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725109761456 time 725121808035 state 0x2 ref 0 00000100:00100000:1.0:1789760795.005751:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.005806:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928448 00000100:00100000:0.0:1789760795.005817:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928448 00000100:00100000:0.0:1789760795.005880:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725109761456 time 725121808035 state 0x0 ref 1 00000100:00100000:0.0:1789760795.005895:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725122068451 time 725121808035 state 0x1 ref 1 00000100:00100000:0.0:1789760795.005934:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725122068451 time 725130540179 state 0x0 ref 1 00000100:00100000:0.0:1789760795.005947:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503928448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.005992:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81223680, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bc8 00010000:00010000:0.0:1789760795.006002:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.006017:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.006021:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.006028:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.006046:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.010975:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987137 is committed 00002000:00010000:2.0:1789760795.011126:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81223680 -> 81227776 00002000:00010000:2.0:1789760795.011131:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158664 -> 158672 00010000:00010000:2.0:1789760795.011204:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.011218:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.011603:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.011611:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.011616:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.011626:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.011645:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.011654:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2643 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81223680->81227775] (req 81223680->81227775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.012856:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503928448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6905us (7107us total) trans 4294987137 rc 0/0 00000100:00100000:2.0:1789760795.012867:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19841 00000100:00100000:2.0:1789760795.012877:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725122068451 time 725130540179 state 0x2 ref 0 00000100:00100000:0.0:1789760795.018708:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.018828:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928576 00000100:00100000:2.0:1789760795.018839:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928576 00000100:00100000:2.0:1789760795.018893:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725122068451 time 725130540179 state 0x0 ref 1 00000100:00100000:2.0:1789760795.018907:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725130800595 time 725130540179 state 0x1 ref 1 00000100:00100000:2.0:1789760795.018946:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725130800595 time 725143552849 state 0x0 ref 1 00000100:00100000:2.0:1789760795.018959:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503928576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.019009:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81227776, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bd0 00010000:00010000:2.0:1789760795.019018:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.019034:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.019038:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.019045:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.019059:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.028914:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987138 is committed 00002000:00010000:2.0:1789760795.029052:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81227776 -> 81231872 00002000:00010000:2.0:1789760795.029058:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158672 -> 158680 00010000:00010000:2.0:1789760795.029070:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.029083:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.029091:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.029104:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.029108:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.029117:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.029135:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.029143:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f264a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81227776->81231871] (req 81227776->81231871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.029240:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503928576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10269us (10529us total) trans 4294987138 rc 0/0 00000100:00100000:2.0:1789760795.029250:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19842 00000100:00100000:2.0:1789760795.030517:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725130800595 time 725143552849 state 0x2 ref 0 00000100:00100000:0.0:1789760795.037772:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760795.037905:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928704 00000100:00100000:1.0:1789760795.037916:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928704 00000100:00100000:1.0:1789760795.037979:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725130800595 time 725143552849 state 0x0 ref 1 00000100:00100000:1.0:1789760795.037995:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725143813265 time 725143552849 state 0x1 ref 1 00000100:00100000:1.0:1789760795.038030:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725143813265 time 725162636412 state 0x0 ref 1 00000100:00100000:1.0:1789760795.038044:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.038095:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81231872, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bd8 00010000:00010000:1.0:1789760795.038105:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.038121:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.038125:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.038132:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.038146:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.047717:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987139 is committed 00002000:00010000:1.0:1789760795.049634:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81231872 -> 81235968 00002000:00010000:1.0:1789760795.049643:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158680 -> 158688 00010000:00010000:1.0:1789760795.049658:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.049669:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.049677:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.049683:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760795.049688:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.049697:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.049715:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.049723:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2651 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81231872->81235967] (req 81231872->81235967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.049806:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11761us (12040us total) trans 4294987139 rc 0/0 00000100:00100000:1.0:1789760795.049817:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19843 00000100:00100000:1.0:1789760795.049826:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725143813265 time 725162636412 state 0x2 ref 0 00000100:00100000:1.0:1789760795.053039:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.053125:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928832 00000100:00100000:0.0:1789760795.053136:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928832 00000100:00100000:0.0:1789760795.053189:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725143813265 time 725162636412 state 0x0 ref 1 00000100:00100000:0.0:1789760795.053204:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725162896828 time 725162636412 state 0x1 ref 1 00000100:00100000:0.0:1789760795.053237:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725162896828 time 725177843476 state 0x0 ref 1 00000100:00100000:0.0:1789760795.053250:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.053318:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81235968, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26be0 00010000:00010000:0.0:1789760795.053327:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.053343:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.053347:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.053354:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.053371:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.059563:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987140 is committed 00002000:00010000:0.0:1789760795.059684:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81235968 -> 81240064 00002000:00010000:0.0:1789760795.059690:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158688 -> 158696 00010000:00010000:0.0:1789760795.059703:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.059716:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.059725:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.059731:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.059736:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.059744:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.059763:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.059771:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2658 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81235968->81240063] (req 81235968->81240063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.059871:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6619us (6836us total) trans 4294987140 rc 0/0 00000100:00100000:0.0:1789760795.059882:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19844 00000100:00100000:0.0:1789760795.059893:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725162896828 time 725177843476 state 0x2 ref 0 00000100:00100000:0.0:1789760795.064834:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.065362:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503928960 00000100:00100000:2.0:1789760795.065372:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503928960 00000100:00100000:2.0:1789760795.065440:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725162896828 time 725177843476 state 0x0 ref 1 00000100:00100000:2.0:1789760795.065454:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725178103892 time 725177843476 state 0x1 ref 1 00000100:00100000:2.0:1789760795.065493:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725178103892 time 725190099884 state 0x0 ref 1 00000100:00100000:2.0:1789760795.065507:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc13850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.065559:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81240064, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26be8 00010000:00010000:2.0:1789760795.065568:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.065583:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.065588:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.065594:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.065608:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.075861:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987141 is committed 00002000:00010000:2.0:1789760795.075979:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81240064 -> 81244160 00002000:00010000:2.0:1789760795.075983:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158696 -> 158704 00010000:00010000:2.0:1789760795.075996:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.076008:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.076016:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.076029:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.076033:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.076042:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.076062:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.076070:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f265f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81240064->81244159] (req 81240064->81244159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.076206:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc13850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503928960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10697us (11373us total) trans 4294987141 rc 0/0 00000100:00100000:2.0:1789760795.076217:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19845 00000100:00100000:2.0:1789760795.076234:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725178103892 time 725190099884 state 0x2 ref 0 00000100:00100000:2.0:1789760795.080253:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.080857:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929088 00000100:00100000:0.0:1789760795.080868:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929088 00000100:00100000:0.0:1789760795.080923:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725178103892 time 725190099884 state 0x0 ref 1 00000100:00100000:0.0:1789760795.080940:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725190360300 time 725190099884 state 0x1 ref 1 00000100:00100000:0.0:1789760795.081997:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725190360300 time 725206592418 state 0x0 ref 1 00000100:00100000:0.0:1789760795.082031:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503929088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.082099:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81244160, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bf0 00010000:00010000:0.0:1789760795.082109:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.082126:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.082130:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.082136:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.082151:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.090744:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987142 is committed 00002000:00010000:0.0:1789760795.090863:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81244160 -> 81248256 00002000:00010000:0.0:1789760795.090869:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158704 -> 158712 00010000:00010000:0.0:1789760795.090881:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.090893:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.090901:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.090910:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.090914:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.090923:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.090943:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.090950:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2666 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81244160->81248255] (req 81244160->81248255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.092106:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503929088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9351us (11123us total) trans 4294987142 rc 0/0 00000100:00100000:0.0:1789760795.092130:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19846 00000100:00100000:0.0:1789760795.092210:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725190360300 time 725206592418 state 0x2 ref 0 00000100:00100000:1.0:1789760795.098535:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.098808:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929216 00000100:00100000:3.0:1789760795.098817:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929216 00000100:00100000:3.0:1789760795.098870:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725190360300 time 725206592418 state 0x0 ref 1 00000100:00100000:3.0:1789760795.098886:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725206852834 time 725206592418 state 0x1 ref 1 00000100:00100000:3.0:1789760795.098918:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725206852834 time 725223524480 state 0x0 ref 1 00000100:00100000:3.0:1789760795.098931:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503929216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.098978:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81248256, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26bf8 00010000:00010000:3.0:1789760795.098996:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.099011:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.099015:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.099022:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.099034:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.106519:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987143 is committed 00002000:00010000:3.0:1789760795.106678:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81248256 -> 81252352 00002000:00010000:3.0:1789760795.106683:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158712 -> 158720 00010000:00010000:3.0:1789760795.106695:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.106706:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.106714:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.106721:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.106725:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.106733:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.106750:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.106757:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f266d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81248256->81252351] (req 81248256->81252351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.107154:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503929216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8221us (8622us total) trans 4294987143 rc 0/0 00000100:00100000:3.0:1789760795.107165:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19847 00000100:00100000:3.0:1789760795.107175:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725206852834 time 725223524480 state 0x2 ref 0 00000100:00100000:2.0:1789760795.112193:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.112858:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929344 00000100:00100000:0.0:1789760795.112869:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929344 00000100:00100000:0.0:1789760795.112926:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725206852834 time 725223524480 state 0x0 ref 1 00000100:00100000:0.0:1789760795.112940:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725223784896 time 725223524480 state 0x1 ref 1 00000100:00100000:0.0:1789760795.112972:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725223784896 time 725237578691 state 0x0 ref 1 00000100:00100000:0.0:1789760795.112992:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503929344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.113036:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81252352, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c00 00010000:00010000:0.0:1789760795.113045:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.113059:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.113063:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.113069:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.113082:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.120498:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987144 is committed 00002000:00010000:0.0:1789760795.124709:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81252352 -> 81256448 00002000:00010000:0.0:1789760795.124717:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158720 -> 158728 00010000:00010000:0.0:1789760795.124730:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.124743:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.124752:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.124760:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.124764:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.124773:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.124792:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.124799:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2674 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81252352->81256447] (req 81252352->81256447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.124891:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503929344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11904us (12700us total) trans 4294987144 rc 0/0 00000100:00100000:0.0:1789760795.124901:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19848 00000100:00100000:0.0:1789760795.124911:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725223784896 time 725237578691 state 0x2 ref 0 00000100:00100000:3.0:1789760795.128014:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.128210:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929472 00000100:00100000:2.0:1789760795.128220:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929472 00000100:00100000:2.0:1789760795.128275:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725223784896 time 725237578691 state 0x0 ref 1 00000100:00100000:2.0:1789760795.128290:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725237839107 time 725237578691 state 0x1 ref 1 00000100:00100000:2.0:1789760795.128323:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725237839107 time 725252929823 state 0x0 ref 1 00000100:00100000:2.0:1789760795.128335:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503929472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.128380:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81256448, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c08 00010000:00010000:2.0:1789760795.128419:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.128434:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.128439:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.128445:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.128458:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.134841:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987145 is committed 00002000:00010000:0.0:1789760795.135139:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81256448 -> 81260544 00002000:00010000:0.0:1789760795.135144:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158728 -> 158736 00010000:00010000:0.0:1789760795.135155:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135166:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135173:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135179:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.135183:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135247:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135266:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135272:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f267b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81256448->81260543] (req 81256448->81260543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.135464:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503929472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7054us (7382us total) trans 4294987145 rc 0/0 00000100:00100000:0.0:1789760795.135487:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19849 00000100:00100000:0.0:1789760795.135497:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725237839107 time 725252929823 state 0x2 ref 0 00000100:00100000:1.0:1789760795.138838:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.138921:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929600 00000100:00100000:2.0:1789760795.138931:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929600 00000100:00100000:2.0:1789760795.138996:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725237839107 time 725252929823 state 0x0 ref 1 00000100:00100000:2.0:1789760795.139011:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725253190239 time 725252929823 state 0x1 ref 1 00000100:00100000:2.0:1789760795.139039:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725253190239 time 725263645342 state 0x0 ref 1 00000100:00100000:2.0:1789760795.139051:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503929600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.139099:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81260544, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c10 00010000:00010000:2.0:1789760795.139109:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.139124:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.139128:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.139135:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.139148:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.147380:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987146 is committed 00002000:00010000:2.0:1789760795.148182:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81260544 -> 81264640 00002000:00010000:2.0:1789760795.148188:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158736 -> 158744 00010000:00010000:2.0:1789760795.148201:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.148213:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.148221:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.148228:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.148232:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.148245:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.148263:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.148270:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2682 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81260544->81264639] (req 81260544->81264639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.148362:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503929600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9309us (9527us total) trans 4294987146 rc 0/0 00000100:00100000:2.0:1789760795.148373:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19850 00000100:00100000:2.0:1789760795.148385:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725253190239 time 725263645342 state 0x2 ref 0 00000100:00100000:2.0:1789760795.153795:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.153942:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929728 00000100:00100000:3.0:1789760795.153952:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929728 00000100:00100000:3.0:1789760795.154059:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725253190239 time 725263645342 state 0x0 ref 1 00000100:00100000:3.0:1789760795.154074:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725263905758 time 725263645342 state 0x1 ref 1 00000100:00100000:3.0:1789760795.154104:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725263905758 time 725278711185 state 0x0 ref 1 00000100:00100000:3.0:1789760795.154260:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503929728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.154307:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81264640, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c18 00010000:00010000:3.0:1789760795.154315:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.154335:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.154341:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.154346:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.154363:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.154986:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760795.162523:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987147 is committed 00002000:00010000:3.0:1789760795.163075:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81264640 -> 81268736 00002000:00010000:3.0:1789760795.163081:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158744 -> 158752 00010000:00010000:3.0:1789760795.163094:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.163107:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.163117:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.163129:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.163207:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.163218:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.163245:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.163253:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2689 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81264640->81268735] (req 81264640->81268735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.163444:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503929728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9324us (9651us total) trans 4294987147 rc 0/0 00000100:00100000:3.0:1789760795.163455:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19851 00000100:00100000:3.0:1789760795.163465:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725263905758 time 725278711185 state 0x2 ref 0 00000100:00100000:3.0:1789760795.168835:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.168941:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929856 00000100:00100000:0.0:1789760795.168951:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929856 00000100:00100000:0.0:1789760795.169006:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725263905758 time 725278711185 state 0x0 ref 1 00000100:00100000:0.0:1789760795.169021:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725278971601 time 725278711185 state 0x1 ref 1 00000100:00100000:0.0:1789760795.169052:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725278971601 time 725293658374 state 0x0 ref 1 00000100:00100000:0.0:1789760795.169064:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503929856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.169107:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81268736, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c20 00010000:00010000:0.0:1789760795.169117:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.169296:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.169303:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.169309:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.169324:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.174648:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987148 is committed 00002000:00010000:0.0:1789760795.174843:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81268736 -> 81272832 00002000:00010000:0.0:1789760795.174849:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158752 -> 158760 00010000:00010000:0.0:1789760795.174864:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.174877:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.174886:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.174893:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.174898:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.174907:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.174926:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.174933:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2690 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81268736->81272831] (req 81268736->81272831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.175075:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503929856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6008us (6244us total) trans 4294987148 rc 0/0 00000100:00100000:0.0:1789760795.175085:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19852 00000100:00100000:0.0:1789760795.175095:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725278971601 time 725293658374 state 0x2 ref 0 00000100:00100000:1.0:1789760795.178650:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.178737:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503929984 00000100:00100000:0.0:1789760795.178747:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503929984 00000100:00100000:0.0:1789760795.178797:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725278971601 time 725293658374 state 0x0 ref 1 00000100:00100000:0.0:1789760795.178809:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725293918790 time 725293658374 state 0x1 ref 1 00000100:00100000:0.0:1789760795.179085:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725293918790 time 725303690589 state 0x0 ref 1 00000100:00100000:0.0:1789760795.179100:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503929984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.179220:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81272832, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c28 00010000:00010000:0.0:1789760795.179230:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.179244:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.179248:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.179254:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.179267:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.186984:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987149 is committed 00002000:00010000:0.0:1789760795.187143:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81272832 -> 81276928 00002000:00010000:0.0:1789760795.187148:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158760 -> 158768 00010000:00010000:0.0:1789760795.187160:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.187179:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.187198:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.187206:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.187210:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.187218:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.187238:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.187244:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2697 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81272832->81276927] (req 81272832->81276927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.187334:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503929984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8233us (8688us total) trans 4294987149 rc 0/0 00000100:00100000:0.0:1789760795.187345:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19853 00000100:00100000:0.0:1789760795.187355:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725293918790 time 725303690589 state 0x2 ref 0 00000100:00100000:2.0:1789760795.189962:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.193659:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503930112 00000100:00100000:0.0:1789760795.193674:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503930112 00000100:00100000:0.0:1789760795.193738:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725293918790 time 725303690589 state 0x0 ref 1 00000100:00100000:0.0:1789760795.193752:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725303951005 time 725303690589 state 0x1 ref 1 00000100:00100000:0.0:1789760795.193852:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725303951005 time 725318456867 state 0x0 ref 1 00000100:00100000:0.0:1789760795.193867:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503930112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.193917:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81276928, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c30 00010000:00010000:0.0:1789760795.193927:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.193942:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.193947:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.193962:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.193984:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.199878:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987150 is committed 00002000:00010000:0.0:1789760795.200385:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81276928 -> 81281024 00002000:00010000:0.0:1789760795.200421:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158768 -> 158776 00010000:00010000:0.0:1789760795.200434:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.200446:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.200461:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.201573:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.201581:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.201594:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.201614:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.201623:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f269e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81276928->81281023] (req 81276928->81281023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.201718:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503930112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7851us (11762us total) trans 4294987150 rc 0/0 00000100:00100000:0.0:1789760795.201727:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19854 00000100:00100000:0.0:1789760795.201736:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725303951005 time 725318456867 state 0x2 ref 0 00000100:00100000:3.0:1789760795.206800:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760795.207213:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503930240 00000100:00100000:1.0:1789760795.207223:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503930240 00000100:00100000:1.0:1789760795.207288:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725303951005 time 725318456867 state 0x0 ref 1 00000100:00100000:1.0:1789760795.207303:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725318717283 time 725318456867 state 0x1 ref 1 00000100:00100000:1.0:1789760795.207370:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725318717283 time 725331975487 state 0x0 ref 1 00000100:00100000:1.0:1789760795.207383:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503930240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.207458:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81281024, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c38 00010000:00010000:1.0:1789760795.207476:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.207492:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.207496:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.207502:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.207516:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.213640:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987151 is committed 00002000:00010000:2.0:1789760795.213807:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81281024 -> 81285120 00002000:00010000:2.0:1789760795.213813:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158776 -> 158784 00010000:00010000:2.0:1789760795.213827:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.213841:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.213851:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.213860:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.213865:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.213874:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.213895:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.213903:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f26a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81281024->81285119] (req 81281024->81285119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.213998:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503930240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6612us (7202us total) trans 4294987151 rc 0/0 00000100:00100000:2.0:1789760795.214009:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19855 00000100:00100000:2.0:1789760795.214020:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725318717283 time 725331975487 state 0x2 ref 0 00000100:00100000:1.0:1789760795.216735:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.216835:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503930368 00000100:00100000:2.0:1789760795.216846:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503930368 00000100:00100000:2.0:1789760795.216896:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725318717283 time 725331975487 state 0x0 ref 1 00000100:00100000:2.0:1789760795.216909:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725332235903 time 725331975487 state 0x1 ref 1 00000100:00100000:2.0:1789760795.216981:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725332235903 time 725341585967 state 0x0 ref 1 00000100:00100000:2.0:1789760795.216997:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503930368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.217045:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81285120, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c40 00010000:00010000:2.0:1789760795.217054:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.217071:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.217076:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.217084:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.217099:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.224635:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987152 is committed 00002000:00010000:3.0:1789760795.229756:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81285120 -> 81289216 00002000:00010000:3.0:1789760795.229765:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158784 -> 158792 00010000:00010000:3.0:1789760795.229778:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.229790:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.229799:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.229806:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.229811:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.229819:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.229837:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.229844:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f26ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81285120->81289215] (req 81285120->81289215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.230018:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503930368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13018us (13285us total) trans 4294987152 rc 0/0 00000100:00100000:3.0:1789760795.230029:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19856 00000100:00100000:3.0:1789760795.230039:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725332235903 time 725341585967 state 0x2 ref 0 00000100:00100000:0.0:1789760795.235335:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.235728:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503930496 00000100:00100000:2.0:1789760795.235858:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503930496 00000100:00100000:2.0:1789760795.235916:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725332235903 time 725341585967 state 0x0 ref 1 00000100:00100000:2.0:1789760795.235939:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725341846383 time 725341585967 state 0x1 ref 1 00000100:00100000:2.0:1789760795.235970:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725341846383 time 725360576450 state 0x0 ref 1 00000100:00100000:2.0:1789760795.235984:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503930496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.236027:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81289216, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c48 00010000:00010000:2.0:1789760795.236036:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.236057:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.236062:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.236069:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.236082:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.242623:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987153 is committed 00002000:00010000:3.0:1789760795.243199:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81289216 -> 81293312 00002000:00010000:3.0:1789760795.243205:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158792 -> 158800 00010000:00010000:3.0:1789760795.243219:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.243233:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.243243:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.243255:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.243260:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.243268:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.243287:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.243294:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f26b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81289216->81293311] (req 81289216->81293311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.243486:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503930496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7499us (8156us total) trans 4294987153 rc 0/0 00000100:00100000:3.0:1789760795.243496:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19857 00000100:00100000:3.0:1789760795.243517:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725341846383 time 725360576450 state 0x2 ref 0 00000100:00100000:3.0:1789760795.248922:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.248996:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503930624 00000100:00100000:0.0:1789760795.249007:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503930624 00000100:00100000:0.0:1789760795.249061:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725341846383 time 725360576450 state 0x0 ref 1 00000100:00100000:0.0:1789760795.249074:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725360836866 time 725360576450 state 0x1 ref 1 00000100:00100000:0.0:1789760795.249103:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725360836866 time 725373710543 state 0x0 ref 1 00000100:00100000:0.0:1789760795.249115:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503930624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.249164:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81293312, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c50 00010000:00010000:0.0:1789760795.249173:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.249188:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.249192:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.249198:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.249211:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.254311:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987154 is committed 00002000:00010000:0.0:1789760795.254605:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81293312 -> 81297408 00002000:00010000:0.0:1789760795.254611:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158800 -> 158808 00010000:00010000:0.0:1789760795.254625:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.254636:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.254650:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.254657:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.254662:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.254670:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.254690:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.254699:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81293312->81297407] (req 81293312->81297407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.254825:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503930624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5707us (5906us total) trans 4294987154 rc 0/0 00000100:00100000:0.0:1789760795.254835:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19858 00000100:00100000:0.0:1789760795.254845:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725360836866 time 725373710543 state 0x2 ref 0 00000100:00100000:1.0:1789760795.264020:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.264164:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503930752 00000100:00100000:2.0:1789760795.264173:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503930752 00000100:00100000:2.0:1789760795.264232:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725360836866 time 725373710543 state 0x0 ref 1 00000100:00100000:2.0:1789760795.264246:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725373970959 time 725373710543 state 0x1 ref 1 00000100:00100000:2.0:1789760795.264271:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725373970959 time 725388878792 state 0x0 ref 1 00000100:00100000:2.0:1789760795.264282:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503930752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.264324:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81297408, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c58 00010000:00010000:2.0:1789760795.264333:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.264347:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.264351:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.264356:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.264370:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.269118:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987155 is committed 00002000:00010000:2.0:1789760795.269233:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81297408 -> 81301504 00002000:00010000:2.0:1789760795.269239:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158808 -> 158816 00010000:00010000:2.0:1789760795.269250:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.269263:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.269273:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.269280:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.269285:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.269295:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.269313:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.269321:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f26c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81297408->81301503] (req 81297408->81301503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.270203:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503930752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5913us (6183us total) trans 4294987155 rc 0/0 00000100:00100000:2.0:1789760795.270217:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19859 00000100:00100000:2.0:1789760795.270311:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725373970959 time 725388878792 state 0x2 ref 0 00000100:00100000:2.0:1789760795.287931:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.290753:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503930880 00000100:00100000:3.0:1789760795.290835:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503930880 00000100:00100000:3.0:1789760795.290899:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725373970959 time 725388878792 state 0x0 ref 1 00000100:00100000:3.0:1789760795.290915:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725389139208 time 725388878792 state 0x1 ref 1 00000100:00100000:3.0:1789760795.290999:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725389139208 time 725415605607 state 0x0 ref 1 00000100:00100000:3.0:1789760795.291015:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503930880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.291066:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81301504, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c60 00010000:00010000:3.0:1789760795.291076:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.291093:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.291098:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.291103:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.291116:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.295698:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987156 is committed 00002000:00010000:3.0:1789760795.295893:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81301504 -> 81305600 00002000:00010000:3.0:1789760795.295899:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158816 -> 158824 00010000:00010000:3.0:1789760795.295911:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.295924:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.296030:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.296039:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.296044:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.296053:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.296072:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.296081:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f26c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81301504->81305599] (req 81301504->81305599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.296190:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503930880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5174us (8264us total) trans 4294987156 rc 0/0 00000100:00100000:3.0:1789760795.296201:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19860 00000100:00100000:3.0:1789760795.296210:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725389139208 time 725415605607 state 0x2 ref 0 00000100:00100000:2.0:1789760795.301789:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.302108:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931008 00000100:00100000:3.0:1789760795.302117:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931008 00000100:00100000:3.0:1789760795.302175:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725389139208 time 725415605607 state 0x0 ref 1 00000100:00100000:3.0:1789760795.302198:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725415866023 time 725415605607 state 0x1 ref 1 00000100:00100000:3.0:1789760795.302227:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725415866023 time 725426833718 state 0x0 ref 1 00000100:00100000:3.0:1789760795.302240:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503931008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.302290:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81305600, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c68 00010000:00010000:3.0:1789760795.302299:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.302314:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.302318:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.302324:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.302339:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.308572:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987157 is committed 00002000:00010000:3.0:1789760795.309017:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81305600 -> 81309696 00002000:00010000:3.0:1789760795.309024:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158824 -> 158832 00010000:00010000:3.0:1789760795.309037:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.309050:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.309059:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.309066:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.309070:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.309079:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.309099:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.309106:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81305600->81309695] (req 81305600->81309695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.309193:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503931008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6950us (7409us total) trans 4294987157 rc 0/0 00000100:00100000:3.0:1789760795.309204:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19861 00000100:00100000:3.0:1789760795.309214:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725415866023 time 725426833718 state 0x2 ref 0 00000100:00100000:1.0:1789760795.315826:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.315913:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931136 00000100:00100000:0.0:1789760795.315923:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931136 00000100:00100000:0.0:1789760795.315985:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725415866023 time 725426833718 state 0x0 ref 1 00000100:00100000:0.0:1789760795.315999:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725427094134 time 725426833718 state 0x1 ref 1 00000100:00100000:0.0:1789760795.316029:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725427094134 time 725440634695 state 0x0 ref 1 00000100:00100000:0.0:1789760795.316042:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503931136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.316085:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81309696, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c70 00010000:00010000:0.0:1789760795.316095:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.316110:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.316115:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.316121:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.316134:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.320823:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987158 is committed 00002000:00010000:2.0:1789760795.320958:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81309696 -> 81313792 00002000:00010000:2.0:1789760795.320963:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158832 -> 158840 00010000:00010000:2.0:1789760795.320976:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.320989:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.320998:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.321006:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.321011:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.321020:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.321043:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.321051:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81309696->81313791] (req 81309696->81313791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.321193:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503931136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5142us (5365us total) trans 4294987158 rc 0/0 00000100:00100000:2.0:1789760795.321204:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19862 00000100:00100000:2.0:1789760795.321214:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725427094134 time 725440634695 state 0x2 ref 0 00000100:00100000:2.0:1789760795.324953:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760795.325056:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931264 00000100:00100000:1.0:1789760795.325064:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931264 00000100:00100000:1.0:1789760795.325115:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725427094134 time 725440634695 state 0x0 ref 1 00000100:00100000:1.0:1789760795.325132:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725440895111 time 725440634695 state 0x1 ref 1 00000100:00100000:1.0:1789760795.325194:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725440895111 time 725449799739 state 0x0 ref 1 00000100:00100000:1.0:1789760795.325216:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503931264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.325270:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81313792, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c78 00010000:00010000:1.0:1789760795.325281:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.325296:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.325300:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.325306:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.325319:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.335352:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987159 is committed 00002000:00010000:1.0:1789760795.335632:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81313792 -> 81317888 00002000:00010000:1.0:1789760795.335638:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158840 -> 158848 00010000:00010000:1.0:1789760795.335653:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.335665:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.335674:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.335681:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760795.335686:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.335695:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.335714:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.335722:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f26dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81313792->81317887] (req 81313792->81317887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.337358:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc9de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503931264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12131us (12401us total) trans 4294987159 rc 0/0 00000100:00100000:1.0:1789760795.337373:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19863 00000100:00100000:1.0:1789760795.337384:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725440895111 time 725449799739 state 0x2 ref 0 00000100:00100000:2.0:1789760795.344063:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.344160:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931392 00000100:00100000:3.0:1789760795.344170:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931392 00000100:00100000:3.0:1789760795.344225:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725440895111 time 725449799739 state 0x0 ref 1 00000100:00100000:3.0:1789760795.344241:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725450060155 time 725449799739 state 0x1 ref 1 00000100:00100000:3.0:1789760795.344302:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725450060155 time 725468908388 state 0x0 ref 1 00000100:00100000:3.0:1789760795.344315:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503931392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.344360:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81317888, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c80 00010000:00010000:3.0:1789760795.344369:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.344385:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.344423:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.344430:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.344444:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.349791:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987160 is committed 00002000:00010000:2.0:1789760795.355640:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81317888 -> 81321984 00002000:00010000:2.0:1789760795.355651:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158848 -> 158856 00010000:00010000:2.0:1789760795.355666:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.355678:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.355687:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.355694:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.355698:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.355707:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.355726:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.355733:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f26e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81317888->81321983] (req 81317888->81321983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.355849:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccea0d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503931392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11531us (11790us total) trans 4294987160 rc 0/0 00000100:00100000:2.0:1789760795.355860:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19864 00000100:00100000:2.0:1789760795.355871:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725450060155 time 725468908388 state 0x2 ref 0 00000100:00100000:3.0:1789760795.360704:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.362503:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931520 00000100:00100000:0.0:1789760795.362517:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931520 00000100:00100000:0.0:1789760795.362859:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725450060155 time 725468908388 state 0x0 ref 1 00000100:00100000:0.0:1789760795.362874:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725469168804 time 725468908388 state 0x1 ref 1 00000100:00100000:0.0:1789760795.362911:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725469168804 time 725487517150 state 0x0 ref 1 00000100:00100000:0.0:1789760795.362925:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503931520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.364669:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81321984, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c88 00010000:00010000:0.0:1789760795.364689:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.364707:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.364712:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.364719:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.364734:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.371232:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987161 is committed 00002000:00010000:0.0:1789760795.371338:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81321984 -> 81326080 00002000:00010000:0.0:1789760795.371344:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158856 -> 158864 00010000:00010000:0.0:1789760795.371358:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.371372:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.371381:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.371412:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.371417:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.371424:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.371442:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.371449:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f26eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81321984->81326079] (req 81321984->81326079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.371546:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503931520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8620us (10846us total) trans 4294987161 rc 0/0 00000100:00100000:0.0:1789760795.371556:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19865 00000100:00100000:0.0:1789760795.371566:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725469168804 time 725487517150 state 0x2 ref 0 00000100:00100000:0.0:1789760795.384328:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.384429:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931648 00000100:00100000:2.0:1789760795.384438:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931648 00000100:00100000:2.0:1789760795.384502:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725469168804 time 725487517150 state 0x0 ref 1 00000100:00100000:2.0:1789760795.384516:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725487777566 time 725487517150 state 0x1 ref 1 00000100:00100000:2.0:1789760795.384545:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725487777566 time 725509152334 state 0x0 ref 1 00000100:00100000:2.0:1789760795.384557:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503931648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.384606:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81326080, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c90 00010000:00010000:2.0:1789760795.384614:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.384627:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.384632:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.384638:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.384657:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.395885:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987162 is committed 00002000:00010000:0.0:1789760795.396029:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81326080 -> 81330176 00002000:00010000:0.0:1789760795.396035:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158864 -> 158872 00010000:00010000:0.0:1789760795.396047:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.396060:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.396071:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.396079:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.396084:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.396094:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.396115:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.396123:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f26f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81326080->81330175] (req 81326080->81330175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.396342:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503931648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11781us (12017us total) trans 4294987162 rc 0/0 00000100:00100000:0.0:1789760795.396353:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19866 00000100:00100000:0.0:1789760795.396362:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725487777566 time 725509152334 state 0x2 ref 0 00000100:00100000:2.0:1789760795.400689:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760795.400795:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931776 00000100:00100000:1.0:1789760795.400805:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931776 00000100:00100000:1.0:1789760795.400857:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725487777566 time 725509152334 state 0x0 ref 1 00000100:00100000:1.0:1789760795.400872:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725509412750 time 725509152334 state 0x1 ref 1 00000100:00100000:1.0:1789760795.400941:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725509412750 time 725525546308 state 0x0 ref 1 00000100:00100000:1.0:1789760795.400954:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503931776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.401000:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81330176, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26c98 00010000:00010000:1.0:1789760795.401009:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.401024:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.401029:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.401035:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.401049:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.408630:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987163 is committed 00002000:00010000:1.0:1789760795.408748:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81330176 -> 81334272 00002000:00010000:1.0:1789760795.408754:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158872 -> 158880 00010000:00010000:1.0:1789760795.408766:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.408779:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.408788:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.408796:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760795.408800:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.408809:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.408828:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.408840:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f26f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81330176->81334271] (req 81330176->81334271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.408921:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503931776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7966us (8240us total) trans 4294987163 rc 0/0 00000100:00100000:1.0:1789760795.408930:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19867 00000100:00100000:1.0:1789760795.408939:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725509412750 time 725525546308 state 0x2 ref 0 00000100:00100000:1.0:1789760795.421514:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.422508:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503931904 00000100:00100000:0.0:1789760795.422519:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503931904 00000100:00100000:0.0:1789760795.422578:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725509412750 time 725525546308 state 0x0 ref 1 00000100:00100000:0.0:1789760795.422593:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725525806724 time 725525546308 state 0x1 ref 1 00000100:00100000:0.0:1789760795.422629:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725525806724 time 725547235913 state 0x0 ref 1 00000100:00100000:0.0:1789760795.422642:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503931904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.422689:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81334272, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26ca0 00010000:00010000:0.0:1789760795.422697:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.422712:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.422716:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.422722:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.422735:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.436198:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987164 is committed 00002000:00010000:0.0:1789760795.436661:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81334272 -> 81338368 00002000:00010000:0.0:1789760795.436667:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158880 -> 158888 00010000:00010000:0.0:1789760795.436681:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.436693:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.436702:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.436715:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.436720:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.436728:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.436749:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.436756:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2700 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81334272->81338367] (req 81334272->81338367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.436853:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503931904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14208us (15345us total) trans 4294987164 rc 0/0 00000100:00100000:0.0:1789760795.436865:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19868 00000100:00100000:0.0:1789760795.436876:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725525806724 time 725547235913 state 0x2 ref 0 00000100:00100000:0.0:1789760795.445661:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.445837:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932032 00000100:00100000:3.0:1789760795.445847:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932032 00000100:00100000:3.0:1789760795.445896:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725525806724 time 725547235913 state 0x0 ref 1 00000100:00100000:3.0:1789760795.445912:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725547496329 time 725547235913 state 0x1 ref 1 00000100:00100000:3.0:1789760795.445943:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725547496329 time 725570550339 state 0x0 ref 1 00000100:00100000:3.0:1789760795.445955:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503932032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.445999:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81338368, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26ca8 00010000:00010000:3.0:1789760795.446008:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.446024:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.446029:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.446035:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.446050:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.452439:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987165 is committed 00002000:00010000:3.0:1789760795.452920:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81338368 -> 81342464 00002000:00010000:3.0:1789760795.452925:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158888 -> 158896 00010000:00010000:3.0:1789760795.452939:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.452952:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.452959:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.452965:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.452969:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.452978:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.452996:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.453004:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2707 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81338368->81342463] (req 81338368->81342463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.454301:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503932032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8338us (8641us total) trans 4294987165 rc 0/0 00000100:00100000:3.0:1789760795.454315:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19869 00000100:00100000:3.0:1789760795.454324:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725547496329 time 725570550339 state 0x2 ref 0 00000100:00100000:2.0:1789760795.459508:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.459599:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932160 00000100:00100000:0.0:1789760795.459609:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932160 00000100:00100000:0.0:1789760795.459663:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725547496329 time 725570550339 state 0x0 ref 1 00000100:00100000:0.0:1789760795.459679:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725570810755 time 725570550339 state 0x1 ref 1 00000100:00100000:0.0:1789760795.459714:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725570810755 time 725584320426 state 0x0 ref 1 00000100:00100000:0.0:1789760795.459726:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503932160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.459773:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81342464, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cb0 00010000:00010000:0.0:1789760795.459782:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.459800:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.459805:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.459811:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.459825:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.466992:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987166 is committed 00002000:00010000:0.0:1789760795.467146:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81342464 -> 81346560 00002000:00010000:0.0:1789760795.467152:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158896 -> 158904 00010000:00010000:0.0:1789760795.467164:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.467175:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.467183:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.467195:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.467199:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.467207:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.467226:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.467234:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f270e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81342464->81346559] (req 81342464->81346559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.467382:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503932160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7653us (7880us total) trans 4294987166 rc 0/0 00000100:00100000:0.0:1789760795.467435:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19870 00000100:00100000:0.0:1789760795.467446:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725570810755 time 725584320426 state 0x2 ref 0 00000100:00100000:1.0:1789760795.472130:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.472322:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932288 00000100:00100000:3.0:1789760795.472331:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932288 00000100:00100000:3.0:1789760795.472378:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725570810755 time 725584320426 state 0x0 ref 1 00000100:00100000:3.0:1789760795.472433:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725584580842 time 725584320426 state 0x1 ref 1 00000100:00100000:2.0:1789760795.472537:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725584580842 time 725597141090 state 0x0 ref 1 00000100:00100000:2.0:1789760795.472690:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503932288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.472740:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81346560, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cb8 00010000:00010000:2.0:1789760795.472749:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.472764:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.472769:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.472775:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.472787:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.484376:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987167 is committed 00002000:00010000:2.0:1789760795.484573:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81346560 -> 81350656 00002000:00010000:2.0:1789760795.484578:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158904 -> 158912 00010000:00010000:2.0:1789760795.484591:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.484604:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.484614:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.484621:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.484626:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.484635:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.484654:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.484668:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2715 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81346560->81350655] (req 81346560->81350655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.484928:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503932288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12374us (12804us total) trans 4294987167 rc 0/0 00000100:00100000:2.0:1789760795.484939:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19871 00000100:00100000:2.0:1789760795.484948:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725584580842 time 725597141090 state 0x2 ref 0 00000100:00100000:2.0:1789760795.496188:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.496302:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932416 00000100:00100000:0.0:1789760795.496313:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932416 00000100:00100000:0.0:1789760795.496370:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725584580842 time 725597141090 state 0x0 ref 1 00000100:00100000:0.0:1789760795.496387:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725597401506 time 725597141090 state 0x1 ref 1 00000100:00100000:0.0:1789760795.496438:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725597401506 time 725621044563 state 0x0 ref 1 00000100:00100000:0.0:1789760795.496452:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503932416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.496512:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81350656, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cc0 00010000:00010000:0.0:1789760795.496522:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.496538:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.496542:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.496550:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.496851:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.505176:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987168 is committed 00002000:00010000:0.0:1789760795.505966:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81350656 -> 81354752 00002000:00010000:0.0:1789760795.505973:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158912 -> 158920 00010000:00010000:0.0:1789760795.505987:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.506001:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.506010:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.506017:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.506023:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.506033:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.506053:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.506060:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f271c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81350656->81354751] (req 81350656->81354751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.506167:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503932416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9713us (9984us total) trans 4294987168 rc 0/0 00000100:00100000:0.0:1789760795.506178:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19872 00000100:00100000:0.0:1789760795.506188:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725597401506 time 725621044563 state 0x2 ref 0 00000100:00100000:1.0:1789760795.516681:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.516910:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932544 00000100:00100000:0.0:1789760795.516920:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932544 00000100:00100000:0.0:1789760795.516978:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725597401506 time 725621044563 state 0x0 ref 1 00000100:00100000:0.0:1789760795.516993:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725621304979 time 725621044563 state 0x1 ref 1 00000100:00100000:0.0:1789760795.517027:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725621304979 time 725641633132 state 0x0 ref 1 00000100:00100000:0.0:1789760795.517041:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.517090:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81354752, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cc8 00010000:00010000:0.0:1789760795.517101:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.517117:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.517121:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.517128:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.517141:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.533137:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987169 is committed 00002000:00010000:1.0:1789760795.533252:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81354752 -> 81358848 00002000:00010000:1.0:1789760795.533257:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158920 -> 158928 00010000:00010000:1.0:1789760795.533271:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.533283:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.533291:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.533298:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760795.533302:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.533317:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.533339:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.533346:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2723 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81354752->81358847] (req 81354752->81358847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.533491:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16448us (16816us total) trans 4294987169 rc 0/0 00000100:00100000:1.0:1789760795.533509:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19873 00000100:00100000:1.0:1789760795.533518:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725621304979 time 725641633132 state 0x2 ref 0 00000100:00100000:1.0:1789760795.539576:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.539713:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932672 00000100:00100000:2.0:1789760795.539724:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932672 00000100:00100000:2.0:1789760795.539859:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725621304979 time 725641633132 state 0x0 ref 1 00000100:00100000:2.0:1789760795.539874:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725641893548 time 725641633132 state 0x1 ref 1 00000100:00100000:2.0:1789760795.539948:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725641893548 time 725664554746 state 0x0 ref 1 00000100:00100000:2.0:1789760795.539962:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.540008:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81358848, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cd0 00010000:00010000:2.0:1789760795.540018:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.540033:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.540037:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.540044:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.540057:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.547528:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987170 is committed 00002000:00010000:2.0:1789760795.547639:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81358848 -> 81362944 00002000:00010000:2.0:1789760795.547644:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158928 -> 158936 00010000:00010000:2.0:1789760795.547657:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.547669:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.547677:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.547685:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.547700:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.547710:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.547729:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.547742:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f272a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81358848->81362943] (req 81358848->81362943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.547851:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7886us (8281us total) trans 4294987170 rc 0/0 00000100:00100000:2.0:1789760795.547861:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19874 00000100:00100000:2.0:1789760795.547872:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725641893548 time 725664554746 state 0x2 ref 0 00000100:00100000:0.0:1789760795.553921:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.554008:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932800 00000100:00100000:2.0:1789760795.554017:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932800 00000100:00100000:2.0:1789760795.554073:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725641893548 time 725664554746 state 0x0 ref 1 00000100:00100000:2.0:1789760795.554089:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725664815162 time 725664554746 state 0x1 ref 1 00000100:00100000:2.0:1789760795.554121:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725664815162 time 725678727883 state 0x0 ref 1 00000100:00100000:2.0:1789760795.554134:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.554184:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81362944, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cd8 00010000:00010000:2.0:1789760795.554193:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.554209:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.554213:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.554221:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.554239:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.570531:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987171 is committed 00002000:00010000:0.0:1789760795.570651:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81362944 -> 81367040 00002000:00010000:0.0:1789760795.570656:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158936 -> 158944 00010000:00010000:0.0:1789760795.570669:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.570682:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.570690:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.570697:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.570701:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.570711:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.570731:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.570738:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2731 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81362944->81367039] (req 81362944->81367039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.570832:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16695us (16918us total) trans 4294987171 rc 0/0 00000100:00100000:0.0:1789760795.570843:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19875 00000100:00100000:0.0:1789760795.570853:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725664815162 time 725678727883 state 0x2 ref 0 00000100:00100000:1.0:1789760795.578722:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.579004:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503932928 00000100:00100000:0.0:1789760795.579013:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503932928 00000100:00100000:0.0:1789760795.579069:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725664815162 time 725678727883 state 0x0 ref 1 00000100:00100000:0.0:1789760795.579084:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725678988299 time 725678727883 state 0x1 ref 1 00000100:00100000:0.0:1789760795.579115:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725678988299 time 725703721952 state 0x0 ref 1 00000100:00100000:0.0:1789760795.579127:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.579173:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81367040, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26ce0 00010000:00010000:0.0:1789760795.579182:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.579208:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.579212:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.579219:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.579232:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.589929:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987172 is committed 00002000:00010000:2.0:1789760795.590184:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81367040 -> 81371136 00002000:00010000:2.0:1789760795.590280:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158944 -> 158952 00010000:00010000:2.0:1789760795.590293:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.590305:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.590314:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.590320:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.590325:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.590334:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.590353:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.590360:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2738 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81367040->81371135] (req 81367040->81371135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.590526:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503932928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11395us (11809us total) trans 4294987172 rc 0/0 00000100:00100000:2.0:1789760795.590537:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19876 00000100:00100000:2.0:1789760795.590547:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725678988299 time 725703721952 state 0x2 ref 0 00000100:00100000:1.0:1789760795.594529:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.596260:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933056 00000100:00100000:0.0:1789760795.596275:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933056 00000100:00100000:0.0:1789760795.596339:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725678988299 time 725703721952 state 0x0 ref 1 00000100:00100000:0.0:1789760795.596355:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725703982368 time 725703721952 state 0x1 ref 1 00000100:00100000:0.0:1789760795.596598:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725703982368 time 725721203522 state 0x0 ref 1 00000100:00100000:0.0:1789760795.596614:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503933056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.596674:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81371136, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26ce8 00010000:00010000:0.0:1789760795.596683:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.596700:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.596705:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.596711:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.596732:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.605356:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987173 is committed 00002000:00010000:0.0:1789760795.605606:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81371136 -> 81375232 00002000:00010000:0.0:1789760795.605612:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158952 -> 158960 00010000:00010000:0.0:1789760795.605625:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.605638:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.605648:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.605655:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.605660:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.605668:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.605689:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.605698:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f273f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81371136->81375231] (req 81371136->81375231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.606019:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503933056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9403us (11493us total) trans 4294987173 rc 0/0 00000100:00100000:0.0:1789760795.606031:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19877 00000100:00100000:0.0:1789760795.606041:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725703982368 time 725721203522 state 0x2 ref 0 00000100:00100000:0.0:1789760795.617818:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.619660:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933184 00000100:00100000:2.0:1789760795.619680:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933184 00000100:00100000:2.0:1789760795.619758:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725703982368 time 725721203522 state 0x0 ref 1 00000100:00100000:2.0:1789760795.619777:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725721463938 time 725721203522 state 0x1 ref 1 00000100:00100000:2.0:1789760795.619828:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725721463938 time 725744424192 state 0x0 ref 1 00000100:00100000:2.0:1789760795.619844:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.619911:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81375232, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cf0 00010000:00010000:2.0:1789760795.619924:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.619940:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.619945:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.619951:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.619965:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.632852:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987174 is committed 00002000:00010000:0.0:1789760795.633170:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81375232 -> 81379328 00002000:00010000:0.0:1789760795.633176:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158960 -> 158968 00010000:00010000:0.0:1789760795.633188:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.633331:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.633340:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.633348:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.633352:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.633362:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.633381:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.633409:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2746 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81375232->81379327] (req 81375232->81379327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.633517:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13672us (15760us total) trans 4294987174 rc 0/0 00000100:00100000:0.0:1789760795.633527:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19878 00000100:00100000:0.0:1789760795.633538:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725721463938 time 725744424192 state 0x2 ref 0 00000100:00100000:1.0:1789760795.637798:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.637888:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933312 00000100:00100000:3.0:1789760795.637899:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933312 00000100:00100000:3.0:1789760795.637974:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725721463938 time 725744424192 state 0x0 ref 1 00000100:00100000:3.0:1789760795.637989:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725744684608 time 725744424192 state 0x1 ref 1 00000100:00100000:3.0:1789760795.638017:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725744684608 time 725762623659 state 0x0 ref 1 00000100:00100000:3.0:1789760795.638029:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.638815:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81379328, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26cf8 00010000:00010000:1.0:1789760795.638828:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.638844:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.638849:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.638856:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.638872:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.645664:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987175 is committed 00002000:00010000:1.0:1789760795.645790:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81379328 -> 81383424 00002000:00010000:1.0:1789760795.645795:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158968 -> 158976 00010000:00010000:1.0:1789760795.645811:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.645824:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.645834:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.645841:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760795.645846:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.645855:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.645876:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.645888:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f274d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81379328->81383423] (req 81379328->81383423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.645988:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7956us (8196us total) trans 4294987175 rc 0/0 00000100:00100000:1.0:1789760795.646003:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19879 00000100:00100000:1.0:1789760795.646014:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725744684608 time 725762623659 state 0x2 ref 0 00000100:00100000:1.0:1789760795.648698:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.648760:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933440 00000100:00100000:2.0:1789760795.648770:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933440 00000100:00100000:2.0:1789760795.648833:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725744684608 time 725762623659 state 0x0 ref 1 00000100:00100000:2.0:1789760795.648847:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725762884075 time 725762623659 state 0x1 ref 1 00000100:00100000:2.0:1789760795.648866:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725762884075 time 725773473986 state 0x0 ref 1 00000100:00100000:2.0:1789760795.648876:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.648918:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81383424, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d00 00010000:00010000:2.0:1789760795.648927:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.648942:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.648946:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.648953:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.648966:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.662445:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987176 is committed 00002000:00010000:2.0:1789760795.662564:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81383424 -> 81387520 00002000:00010000:2.0:1789760795.662569:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158976 -> 158984 00010000:00010000:2.0:1789760795.662582:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.662595:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.662603:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.662610:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.662614:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.662622:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.662641:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.662648:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2754 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81383424->81387519] (req 81383424->81387519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.662983:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14102us (14288us total) trans 4294987176 rc 0/0 00000100:00100000:2.0:1789760795.662993:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19880 00000100:00100000:2.0:1789760795.663002:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725762884075 time 725773473986 state 0x2 ref 0 00000100:00100000:0.0:1789760795.675484:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760795.675601:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933568 00000100:00100000:1.0:1789760795.675611:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933568 00000100:00100000:1.0:1789760795.675665:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725762884075 time 725773473986 state 0x0 ref 1 00000100:00100000:1.0:1789760795.675679:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725773734402 time 725773473986 state 0x1 ref 1 00000100:00100000:1.0:1789760795.676187:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725773734402 time 725800378302 state 0x0 ref 1 00000100:00100000:1.0:1789760795.676207:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503933568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.676254:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81387520, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d08 00010000:00010000:1.0:1789760795.676261:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.676274:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.676277:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.676283:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.676292:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.684739:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987177 is committed 00002000:00010000:1.0:1789760795.684885:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81387520 -> 81391616 00002000:00010000:1.0:1789760795.684891:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158984 -> 158992 00010000:00010000:1.0:1789760795.684905:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.684918:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.684926:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.684933:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760795.684937:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.684946:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.684965:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.684992:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f275b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81387520->81391615] (req 81387520->81391615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.685475:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6738000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503933568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9259us (9997us total) trans 4294987177 rc 0/0 00000100:00100000:1.0:1789760795.685487:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19881 00000100:00100000:1.0:1789760795.685497:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725773734402 time 725800378302 state 0x2 ref 0 00000100:00100000:1.0:1789760795.690217:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.690311:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933696 00000100:00100000:0.0:1789760795.690323:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933696 00000100:00100000:0.0:1789760795.690384:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725773734402 time 725800378302 state 0x0 ref 1 00000100:00100000:0.0:1789760795.690479:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725800638718 time 725800378302 state 0x1 ref 1 00000100:00100000:0.0:1789760795.690546:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725800638718 time 725815152194 state 0x0 ref 1 00000100:00100000:0.0:1789760795.690559:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.690607:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81391616, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d10 00010000:00010000:0.0:1789760795.690617:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.690633:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.690638:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.690645:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.690660:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.697662:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987178 is committed 00002000:00010000:0.0:1789760795.697777:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81391616 -> 81395712 00002000:00010000:0.0:1789760795.697783:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 158992 -> 159000 00010000:00010000:0.0:1789760795.697797:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.697808:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.697818:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.697825:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.697830:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.697851:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.697873:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.697881:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2762 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81391616->81395711] (req 81391616->81395711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.697983:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503933696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7420us (7769us total) trans 4294987178 rc 0/0 00000100:00100000:0.0:1789760795.697994:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19882 00000100:00100000:0.0:1789760795.698003:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725800638718 time 725815152194 state 0x2 ref 0 00000100:00100000:3.0:1789760795.702097:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.702761:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933824 00000100:00100000:2.0:1789760795.703212:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933824 00000100:00100000:2.0:1789760795.703278:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725800638718 time 725815152194 state 0x0 ref 1 00000100:00100000:2.0:1789760795.703300:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725815412610 time 725815152194 state 0x1 ref 1 00000100:00100000:2.0:1789760795.703341:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725815412610 time 725827947197 state 0x0 ref 1 00000100:00100000:2.0:1789760795.703488:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503933824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.703539:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81395712, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d18 00010000:00010000:2.0:1789760795.703550:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.703566:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760795.703571:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.703579:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.703592:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.711554:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987179 is committed 00002000:00010000:2.0:1789760795.711669:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81395712 -> 81399808 00002000:00010000:2.0:1789760795.711674:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159000 -> 159008 00010000:00010000:2.0:1789760795.711687:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.711705:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.711713:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.711720:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.711725:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.711735:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.711757:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.711764:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2769 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81395712->81399807] (req 81395712->81399807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.711900:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503933824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8543us (9807us total) trans 4294987179 rc 0/0 00000100:00100000:2.0:1789760795.711910:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19883 00000100:00100000:2.0:1789760795.711923:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725815412610 time 725827947197 state 0x2 ref 0 00000100:00100000:2.0:1789760795.716855:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.716917:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503933952 00000100:00100000:3.0:1789760795.716928:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503933952 00000100:00100000:3.0:1789760795.716985:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725815412610 time 725827947197 state 0x0 ref 1 00000100:00100000:3.0:1789760795.717000:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725828207613 time 725827947197 state 0x1 ref 1 00000100:00100000:3.0:1789760795.717023:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725828207613 time 725841628975 state 0x0 ref 1 00000100:00100000:3.0:1789760795.717035:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503933952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.717082:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81399808, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d20 00010000:00010000:3.0:1789760795.717090:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.717104:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.717108:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.717114:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.717128:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.723925:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987180 is committed 00002000:00010000:3.0:1789760795.724189:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81399808 -> 81403904 00002000:00010000:3.0:1789760795.724194:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159008 -> 159016 00010000:00010000:3.0:1789760795.724207:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.724220:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.724229:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.724243:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.724247:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.724256:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.724277:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.724285:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2770 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81399808->81403903] (req 81399808->81403903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.725364:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc95f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503933952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7350us (7539us total) trans 4294987180 rc 0/0 00000100:00100000:3.0:1789760795.725383:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19884 00000100:00100000:3.0:1789760795.725459:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725828207613 time 725841628975 state 0x2 ref 0 00000100:00100000:1.0:1789760795.731696:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.731799:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934080 00000100:00100000:0.0:1789760795.731961:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934080 00000100:00100000:0.0:1789760795.732016:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725828207613 time 725841628975 state 0x0 ref 1 00000100:00100000:0.0:1789760795.732029:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725841889391 time 725841628975 state 0x1 ref 1 00000100:00100000:0.0:1789760795.732100:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725841889391 time 725856706396 state 0x0 ref 1 00000100:00100000:0.0:1789760795.732112:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503934080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.732156:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81403904, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d28 00010000:00010000:0.0:1789760795.732165:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.732179:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.732183:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.732198:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.732211:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.737341:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987181 is committed 00002000:00010000:0.0:1789760795.737475:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81403904 -> 81408000 00002000:00010000:0.0:1789760795.737480:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159016 -> 159024 00010000:00010000:0.0:1789760795.737493:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.737507:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.737516:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.737524:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.737529:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.737539:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.737560:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.737569:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f2777 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81403904->81407999] (req 81403904->81407999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.737703:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503934080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5587us (6011us total) trans 4294987181 rc 0/0 00000100:00100000:0.0:1789760795.737714:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19885 00000100:00100000:0.0:1789760795.737724:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725841889391 time 725856706396 state 0x2 ref 0 00000100:00100000:1.0:1789760795.740430:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.740537:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934208 00000100:00100000:0.0:1789760795.740547:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934208 00000100:00100000:0.0:1789760795.740602:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725841889391 time 725856706396 state 0x0 ref 1 00000100:00100000:0.0:1789760795.740617:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725856966812 time 725856706396 state 0x1 ref 1 00000100:00100000:0.0:1789760795.740648:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725856966812 time 725865254154 state 0x0 ref 1 00000100:00100000:0.0:1789760795.740661:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503934208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.740707:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81408000, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d30 00010000:00010000:0.0:1789760795.740717:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.740732:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.740747:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.740754:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.740772:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.747630:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987182 is committed 00002000:00010000:0.0:1789760795.747750:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81408000 -> 81412096 00002000:00010000:0.0:1789760795.747755:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159024 -> 159032 00010000:00010000:0.0:1789760795.747769:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.747782:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.747791:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.747798:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.747803:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.747812:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.748288:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.748298:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f277e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81408000->81412095] (req 81408000->81412095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.748443:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503934208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7780us (8017us total) trans 4294987182 rc 0/0 00000100:00100000:0.0:1789760795.748455:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19886 00000100:00100000:0.0:1789760795.748476:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725856966812 time 725865254154 state 0x2 ref 0 00000100:00100000:2.0:1789760795.751587:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760795.752185:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934336 00000100:00100000:1.0:1789760795.752197:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934336 00000100:00100000:1.0:1789760795.752270:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725856966812 time 725865254154 state 0x0 ref 1 00000100:00100000:1.0:1789760795.752286:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725865514570 time 725865254154 state 0x1 ref 1 00000100:00100000:1.0:1789760795.752318:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725865514570 time 725876925077 state 0x0 ref 1 00000100:00100000:1.0:1789760795.752331:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503934336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.752377:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81412096, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d38 00010000:00010000:1.0:1789760795.752430:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.752487:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.752492:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.752511:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.752525:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.759032:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987183 is committed 00002000:00010000:2.0:1789760795.759737:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81412096 -> 81416192 00002000:00010000:2.0:1789760795.759744:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159032 -> 159040 00010000:00010000:2.0:1789760795.759956:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759970:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759985:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759993:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.759998:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.760009:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.760029:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.760037:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2785 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81412096->81416191] (req 81412096->81416191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.760130:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503934336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7797us (8549us total) trans 4294987183 rc 0/0 00000100:00100000:2.0:1789760795.760141:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19887 00000100:00100000:2.0:1789760795.760152:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725865514570 time 725876925077 state 0x2 ref 0 00000100:00100000:1.0:1789760795.767819:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.772301:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934464 00000100:00100000:3.0:1789760795.772315:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934464 00000100:00100000:3.0:1789760795.772381:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725865514570 time 725876925077 state 0x0 ref 1 00000100:00100000:3.0:1789760795.772413:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725877185493 time 725876925077 state 0x1 ref 1 00000100:00100000:3.0:1789760795.772437:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725877185493 time 725897043945 state 0x0 ref 1 00000100:00100000:3.0:1789760795.772453:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503934464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.772895:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81416192, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d40 00010000:00010000:3.0:1789760795.772905:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.772920:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.772925:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.772932:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.772945:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.779962:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987184 is committed 00002000:00010000:3.0:1789760795.780077:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81416192 -> 81420288 00002000:00010000:3.0:1789760795.780083:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159040 -> 159048 00010000:00010000:3.0:1789760795.780095:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.780108:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.780118:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.780125:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.780131:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.780140:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.780160:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.780167:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f278c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81416192->81420287] (req 81416192->81420287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.780379:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503934464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7924us (12564us total) trans 4294987184 rc 0/0 00000100:00100000:3.0:1789760795.780438:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19888 00000100:00100000:3.0:1789760795.780449:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725877185493 time 725897043945 state 0x2 ref 0 00000100:00100000:1.0:1789760795.790528:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.790627:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934592 00000100:00100000:0.0:1789760795.790637:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934592 00000100:00100000:0.0:1789760795.790696:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725877185493 time 725897043945 state 0x0 ref 1 00000100:00100000:0.0:1789760795.790711:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725897304361 time 725897043945 state 0x1 ref 1 00000100:00100000:0.0:1789760795.790744:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725897304361 time 725915351069 state 0x0 ref 1 00000100:00100000:0.0:1789760795.790756:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503934592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.790804:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81420288, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x26d48 00010000:00010000:0.0:1789760795.790812:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.790827:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.790832:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.790838:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.790851:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.797432:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987185 is committed 00002000:00010000:0.0:1789760795.797965:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81420288 -> 81424384 00002000:00010000:0.0:1789760795.797971:0:8436:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760795.797973:0:8436:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760795.797975:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159048 -> 159056 00010000:00010000:0.0:1789760795.797987:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.797999:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.798008:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.798014:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.798018:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.798026:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.798045:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.798053:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2793 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81420288->81424383] (req 81420288->81424383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.798208:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503934592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7447us (7683us total) trans 4294987185 rc 0/0 00000100:00100000:0.0:1789760795.798220:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19889 00000100:00100000:0.0:1789760795.798230:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725897304361 time 725915351069 state 0x2 ref 0 00000100:00100000:2.0:1789760795.802988:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.803132:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934720 00000100:00100000:0.0:1789760795.803141:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934720 00000100:00100000:0.0:1789760795.803286:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725897304361 time 725915351069 state 0x0 ref 1 00000100:00100000:0.0:1789760795.803303:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725915611485 time 725915351069 state 0x1 ref 1 00000100:00100000:0.0:1789760795.803362:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725915611485 time 725927968471 state 0x0 ref 1 00000100:00100000:0.0:1789760795.803375:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503934720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.803447:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81424384, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d50 00010000:00010000:0.0:1789760795.803456:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.803481:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760795.803485:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.803492:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.803513:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.808353:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987186 is committed 00002000:00010000:0.0:1789760795.808924:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81424384 -> 81428480 00002000:00010000:0.0:1789760795.808929:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159056 -> 159064 00010000:00010000:0.0:1789760795.808943:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.808955:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.808968:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.808975:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dac00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.808980:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.808989:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.809009:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.809017:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094c2e6f3/0xfa2da65b083f279a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81424384->81428479] (req 81424384->81428479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.809268:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503934720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5890us (6283us total) trans 4294987186 rc 0/0 00000100:00100000:0.0:1789760795.809279:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19890 00000100:00100000:0.0:1789760795.809288:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725915611485 time 725927968471 state 0x2 ref 0 00000100:00100000:1.0:1789760795.813707:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.813831:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934848 00000100:00100000:3.0:1789760795.813841:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934848 00000100:00100000:3.0:1789760795.813895:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725915611485 time 725927968471 state 0x0 ref 1 00000100:00100000:3.0:1789760795.813910:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725928228887 time 725927968471 state 0x1 ref 1 00000100:00100000:3.0:1789760795.813975:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725928228887 time 725938581766 state 0x0 ref 1 00000100:00100000:3.0:1789760795.813988:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503934848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.814034:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81428480, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d58 00010000:00010000:3.0:1789760795.814044:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.814071:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.814076:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.814082:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.814098:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.819873:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987187 is committed 00002000:00010000:2.0:1789760795.819964:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81428480 -> 81432576 00002000:00010000:2.0:1789760795.819969:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159064 -> 159072 00010000:00010000:2.0:1789760795.819979:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819988:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819997:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.820004:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.820009:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.820018:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.820038:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.820050:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f27a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81428480->81432575] (req 81428480->81432575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.820270:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503934848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6278us (6566us total) trans 4294987187 rc 0/0 00000100:00100000:2.0:1789760795.820281:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19891 00000100:00100000:2.0:1789760795.820292:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725928228887 time 725938581766 state 0x2 ref 0 00000100:00100000:1.0:1789760795.823701:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760795.823925:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503934976 00000100:00100000:3.0:1789760795.823936:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503934976 00000100:00100000:3.0:1789760795.823989:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725928228887 time 725938581766 state 0x0 ref 1 00000100:00100000:3.0:1789760795.824003:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725938842182 time 725938581766 state 0x1 ref 1 00000100:00100000:3.0:1789760795.824325:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725938842182 time 725948929207 state 0x0 ref 1 00000100:00100000:3.0:1789760795.824340:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503934976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760795.824501:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81432576, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d60 00010000:00010000:3.0:1789760795.824512:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.824528:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760795.824533:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.824540:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.824616:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.831668:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987188 is committed 00002000:00010000:2.0:1789760795.834737:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81432576 -> 81436672 00002000:00010000:2.0:1789760795.834747:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159072 -> 159080 00010000:00010000:2.0:1789760795.834764:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.834777:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.834786:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.834793:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.834797:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.834977:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.835173:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.835182:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f27a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81432576->81436671] (req 81432576->81436671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.836710:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503934976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12365us (13007us total) trans 4294987188 rc 0/0 00000100:00100000:2.0:1789760795.836723:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19892 00000100:00100000:2.0:1789760795.836734:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725938842182 time 725948929207 state 0x2 ref 0 00000100:00100000:0.0:1789760795.846703:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760795.851667:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503935104 00000100:00100000:1.0:1789760795.852667:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503935104 00000100:00100000:1.0:1789760795.852741:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725938842182 time 725948929207 state 0x0 ref 1 00000100:00100000:1.0:1789760795.853022:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725949189623 time 725948929207 state 0x1 ref 1 00000100:00100000:1.0:1789760795.853068:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725949189623 time 725977674791 state 0x0 ref 1 00000100:00100000:1.0:1789760795.853083:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503935104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760795.853133:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81436672, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d68 00010000:00010000:1.0:1789760795.853143:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.853159:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760795.853163:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.853170:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.853193:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.858781:0:3636:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc743fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523616896:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760795.858862:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760795.858977:0:15166:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523616896 00000100:00100000:0.0:1789760795.858986:0:15166:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523616896 00000100:00100000:2.0:1789760795.859045:0:8432:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1789760795.859064:0:8432:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523616896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760795.859087:0:8432:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3842330624 avail 3609059328 00000100:00100000:2.0:1789760795.859152:0:8432:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7eb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3636:x1876699523616896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760795.859220:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5cc687b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523617024:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760795.859250:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760795.859289:0:8433:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523617024 00000100:00100000:1.0:1789760795.859293:0:8433:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523617024 00000100:00100000:3.0:1789760795.859314:0:3636:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc743fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3636:1876699523616896:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760795.860926:0:8432:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000001:00080000:2.0:1789760795.862173:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987189 is committed 00000100:00100000:1.0:1789760795.863925:0:8433:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760795.863949:0:8433:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffb6b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523617024:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760795.863976:0:8433:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3842330624 avail 3609030656 00000100:00100000:1.0:1789760795.864064:0:8433:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffb6b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1876699523617024:12345-0@lo:13:osp-pre-0-1.0 Request processed in 127us (4813us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760795.864071:0:8433:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:3.0:1789760795.864161:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5cc687b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3637:1876699523617024:0@lo:13:osp-pre-0-1.0 00002000:00010000:0.0:1789760795.869085:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81436672 -> 81440768 00002000:00010000:0.0:1789760795.869095:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159080 -> 159088 00010000:00010000:0.0:1789760795.869110:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.869123:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.869132:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.869140:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.869145:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.869154:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.869173:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.869181:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f27af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81436672->81440767] (req 81436672->81440767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.869272:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503935104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16187us (22572us total) trans 4294987189 rc 0/0 00000100:00100000:0.0:1789760795.869283:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19893 00000100:00100000:0.0:1789760795.869292:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725949189623 time 725977674791 state 0x2 ref 0 00000100:00100000:3.0:1789760795.876952:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.877766:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503935232 00000100:00100000:0.0:1789760795.877777:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503935232 00000100:00100000:0.0:1789760795.877885:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725949189623 time 725977674791 state 0x0 ref 1 00000100:00100000:0.0:1789760795.877902:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725977935207 time 725977674791 state 0x1 ref 1 00000100:00100000:0.0:1789760795.878121:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725977935207 time 726002726195 state 0x0 ref 1 00000100:00100000:0.0:1789760795.878136:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503935232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.878197:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81440768, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d70 00010000:00010000:0.0:1789760795.878207:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.878223:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760795.878238:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.878246:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.878262:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.885089:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987190 is committed 00002000:00010000:2.0:1789760795.885183:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81440768 -> 81444864 00002000:00010000:2.0:1789760795.885237:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159088 -> 159096 00010000:00010000:2.0:1789760795.885252:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885265:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885280:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885287:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.885301:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885312:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885331:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885339:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f27b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81440768->81444863] (req 81440768->81444863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.885594:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503935232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7456us (8646us total) trans 4294987190 rc 0/0 00000100:00100000:2.0:1789760795.885605:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19894 00000100:00100000:2.0:1789760795.885614:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725977935207 time 726002726195 state 0x2 ref 0 00000100:00100000:1.0:1789760795.889289:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.889428:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503935360 00000100:00100000:2.0:1789760795.889438:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503935360 00000100:00100000:2.0:1789760795.889508:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 725977935207 time 726002726195 state 0x0 ref 1 00000100:00100000:2.0:1789760795.889524:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726002986611 time 726002726195 state 0x1 ref 1 00000100:00100000:2.0:1789760795.889547:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726002986611 time 726014154010 state 0x0 ref 1 00000100:00100000:2.0:1789760795.889559:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.889806:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81444864, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d78 00010000:00010000:2.0:1789760795.889823:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.889839:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760795.889843:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.889850:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.889863:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.895883:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987191 is committed 00002000:00010000:2.0:1789760795.903945:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81444864 -> 81448960 00002000:00010000:2.0:1789760795.903954:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159096 -> 159104 00010000:00010000:2.0:1789760795.903971:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.903984:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.903994:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.904007:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.904011:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.904019:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.904037:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.904044:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81444864->81448959] (req 81444864->81448959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.904143:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14582us (14860us total) trans 4294987191 rc 0/0 00000100:00100000:2.0:1789760795.904154:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19895 00000100:00100000:2.0:1789760795.904164:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726002986611 time 726014154010 state 0x2 ref 0 00000100:00100000:0.0:1789760795.908231:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.908311:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503935488 00000100:00100000:2.0:1789760795.908320:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503935488 00000100:00100000:2.0:1789760795.908374:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726002986611 time 726014154010 state 0x0 ref 1 00000100:00100000:2.0:1789760795.908414:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726014414426 time 726014154010 state 0x1 ref 1 00000100:00100000:2.0:1789760795.908435:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726014414426 time 726033042415 state 0x0 ref 1 00000100:00100000:2.0:1789760795.908450:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.908507:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81448960, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d80 00010000:00010000:2.0:1789760795.908516:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.908532:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760795.908536:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.908543:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.908556:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.914022:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987192 is committed 00002000:00010000:3.0:1789760795.914364:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81448960 -> 81453056 00002000:00010000:3.0:1789760795.914370:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159104 -> 159112 00010000:00010000:3.0:1789760795.914383:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.914414:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.914424:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.914432:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760795.914436:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.914445:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.914477:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.914486:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f27c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81448960->81453055] (req 81448960->81453055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.914670:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6217us (6443us total) trans 4294987192 rc 0/0 00000100:00100000:3.0:1789760795.914681:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19896 00000100:00100000:3.0:1789760795.914696:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726014414426 time 726033042415 state 0x2 ref 0 00000100:00100000:3.0:1789760795.926076:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.926192:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503935616 00000100:00100000:0.0:1789760795.926204:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503935616 00000100:00100000:0.0:1789760795.926271:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726014414426 time 726033042415 state 0x0 ref 1 00000100:00100000:0.0:1789760795.926290:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726033302831 time 726033042415 state 0x1 ref 1 00000100:00100000:0.0:1789760795.926323:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726033302831 time 726050929562 state 0x0 ref 1 00000100:00100000:0.0:1789760795.926336:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.926382:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81453056, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d88 00010000:00010000:0.0:1789760795.926430:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.926446:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760795.926450:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.926456:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.926480:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.936110:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987193 is committed 00002000:00010000:0.0:1789760795.936319:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81453056 -> 81457152 00002000:00010000:0.0:1789760795.936326:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159112 -> 159120 00010000:00010000:0.0:1789760795.936341:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.936353:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.936367:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.936375:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760795.936381:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.936488:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.936574:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.936582:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f27cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81453056->81457151] (req 81453056->81457151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.937066:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10728us (10996us total) trans 4294987193 rc 0/0 00000100:00100000:0.0:1789760795.937077:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19897 00000100:00100000:0.0:1789760795.937086:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726033302831 time 726050929562 state 0x2 ref 0 00000100:00100000:1.0:1789760795.944118:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760795.944320:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503935744 00000100:00100000:0.0:1789760795.944329:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503935744 00000100:00100000:0.0:1789760795.944383:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726033302831 time 726050929562 state 0x0 ref 1 00000100:00100000:0.0:1789760795.944427:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726051189978 time 726050929562 state 0x1 ref 1 00000100:00100000:0.0:1789760795.944456:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726051189978 time 726069062694 state 0x0 ref 1 00000100:00100000:0.0:1789760795.944479:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503935744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760795.944525:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81457152, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d90 00010000:00010000:0.0:1789760795.944534:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.944550:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760795.944554:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.944560:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.944574:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.960749:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987194 is committed 00002000:00010000:1.0:1789760795.961097:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81457152 -> 81461248 00002000:00010000:1.0:1789760795.961103:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159120 -> 159128 00010000:00010000:1.0:1789760795.961115:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.961128:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.961136:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.961143:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760795.961147:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.961157:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.961178:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.961245:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f27d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81457152->81461247] (req 81457152->81461247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.961344:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503935744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16863us (17231us total) trans 4294987194 rc 0/0 00000100:00100000:1.0:1789760795.961355:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19898 00000100:00100000:1.0:1789760795.961366:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726051189978 time 726069062694 state 0x2 ref 0 00000100:00100000:0.0:1789760795.965146:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760795.966351:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503935872 00000100:00100000:2.0:1789760795.966364:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503935872 00000100:00100000:2.0:1789760795.966435:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726051189978 time 726069062694 state 0x0 ref 1 00000100:00100000:2.0:1789760795.966450:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726069323110 time 726069062694 state 0x1 ref 1 00000100:00100000:2.0:1789760795.966483:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726069323110 time 726091090317 state 0x0 ref 1 00000100:00100000:2.0:1789760795.966496:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760795.966613:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81461248, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26d98 00010000:00010000:2.0:1789760795.966621:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.966638:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760795.966642:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.966648:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.966663:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.988494:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987195 is committed 00002000:00010000:2.0:1789760795.988691:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81461248 -> 81465344 00002000:00010000:2.0:1789760795.988697:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159128 -> 159136 00010000:00010000:2.0:1789760795.988711:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.988722:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.988731:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.988739:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760795.988744:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.988754:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.988780:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.988789:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81461248->81465343] (req 81461248->81465343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.989895:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503935872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 23389us (24746us total) trans 4294987195 rc 0/0 00000100:00100000:2.0:1789760795.989909:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19899 00000100:00100000:2.0:1789760795.989919:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726069323110 time 726091090317 state 0x2 ref 0 00000100:00100000:3.0:1789760795.999995:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.002619:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936000 00000100:00100000:2.0:1789760796.002633:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936000 00000100:00100000:2.0:1789760796.002715:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726069323110 time 726091090317 state 0x0 ref 1 00000100:00100000:2.0:1789760796.002731:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726091350733 time 726091090317 state 0x1 ref 1 00000100:00080000:1.0:1789760796.002889:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760796.003212:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726091350733 time 726127814974 state 0x0 ref 1 00000100:00100000:2.0:1789760796.003231:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.003285:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81465344, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26da0 00010000:00010000:2.0:1789760796.003293:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.003310:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.003314:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.003320:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.003335:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.012209:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987196 is committed 00002000:00010000:2.0:1789760796.012330:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81465344 -> 81469440 00002000:00010000:2.0:1789760796.012335:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159136 -> 159144 00010000:00010000:2.0:1789760796.012350:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.012362:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.012371:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.012378:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.012383:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.012429:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.012448:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.012471:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81465344->81469439] (req 81465344->81469439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.012555:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9326us (12577us total) trans 4294987196 rc 0/0 00000100:00100000:2.0:1789760796.012564:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19900 00000100:00100000:2.0:1789760796.012574:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726091350733 time 726127814974 state 0x2 ref 0 00000100:00100000:3.0:1789760796.015705:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.016495:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936128 00000100:00100000:0.0:1789760796.016516:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936128 00000100:00100000:0.0:1789760796.016575:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726091350733 time 726127814974 state 0x0 ref 1 00000100:00100000:0.0:1789760796.016590:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726128075390 time 726127814974 state 0x1 ref 1 00000100:00100000:0.0:1789760796.016623:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726128075390 time 726141229610 state 0x0 ref 1 00000100:00100000:0.0:1789760796.016638:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503936128:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.016688:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81469440, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26da8 00010000:00010000:0.0:1789760796.016697:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.016712:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760796.016716:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.016721:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.016735:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.027741:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987197 is committed 00002000:00010000:1.0:1789760796.027856:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81469440 -> 81473536 00002000:00010000:1.0:1789760796.027861:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159144 -> 159152 00010000:00010000:1.0:1789760796.027875:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.027886:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.027894:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.027901:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.027905:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.027914:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.027934:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.027941:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f27e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81469440->81473535] (req 81469440->81473535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.028151:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503936128:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11509us (12449us total) trans 4294987197 rc 0/0 00000100:00100000:1.0:1789760796.028161:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19901 00000100:00100000:1.0:1789760796.028172:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726128075390 time 726141229610 state 0x2 ref 0 00000100:00100000:0.0:1789760796.043295:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.043367:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936256 00000100:00100000:3.0:1789760796.043377:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936256 00000100:00100000:3.0:1789760796.043473:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726128075390 time 726141229610 state 0x0 ref 1 00000100:00100000:3.0:1789760796.043488:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726141490026 time 726141229610 state 0x1 ref 1 00000100:00100000:3.0:1789760796.043563:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726141490026 time 726168169266 state 0x0 ref 1 00000100:00100000:3.0:1789760796.043576:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc11450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503936256:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.043622:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81473536, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26db0 00010000:00010000:3.0:1789760796.043631:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.043646:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.043649:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.043655:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.043667:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.053559:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987198 is committed 00002000:00010000:2.0:1789760796.054324:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81473536 -> 81477632 00002000:00010000:2.0:1789760796.054331:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159152 -> 159160 00010000:00010000:2.0:1789760796.054344:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.054356:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.054364:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.054372:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.054376:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.054384:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.054448:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.054455:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f27ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81473536->81477631] (req 81473536->81477631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.054569:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc11450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503936256:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10990us (11277us total) trans 4294987198 rc 0/0 00000100:00100000:2.0:1789760796.054580:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19902 00000100:00100000:2.0:1789760796.054590:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726141490026 time 726168169266 state 0x2 ref 0 00000100:00100000:1.0:1789760796.057142:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.057231:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936384 00000100:00100000:3.0:1789760796.057242:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936384 00000100:00100000:3.0:1789760796.057301:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726141490026 time 726168169266 state 0x0 ref 1 00000100:00100000:3.0:1789760796.057316:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726168429682 time 726168169266 state 0x1 ref 1 00000100:00100000:3.0:1789760796.057345:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726168429682 time 726181951984 state 0x0 ref 1 00000100:00100000:3.0:1789760796.057361:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936384:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.057450:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81477632, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26db8 00010000:00010000:3.0:1789760796.057468:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.057484:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.057489:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.057496:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.057517:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.064664:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987199 is committed 00002000:00010000:3.0:1789760796.065287:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81477632 -> 81481728 00002000:00010000:3.0:1789760796.065292:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159160 -> 159168 00010000:00010000:3.0:1789760796.065306:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.065318:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.065326:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.065333:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.065338:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.065347:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.065366:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.065374:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f27f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81477632->81481727] (req 81477632->81481727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.065497:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936384:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8137us (8371us total) trans 4294987199 rc 0/0 00000100:00100000:3.0:1789760796.065508:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19903 00000100:00100000:3.0:1789760796.065517:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726168429682 time 726181951984 state 0x2 ref 0 00000100:00100000:3.0:1789760796.070656:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.070797:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936512 00000100:00100000:2.0:1789760796.070962:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936512 00000100:00100000:2.0:1789760796.071020:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726168429682 time 726181951984 state 0x0 ref 1 00000100:00100000:2.0:1789760796.071035:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726182212400 time 726181951984 state 0x1 ref 1 00000100:00100000:2.0:1789760796.071119:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726182212400 time 726195724050 state 0x0 ref 1 00000100:00100000:2.0:1789760796.071132:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936512:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.071181:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81481728, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26dc0 00010000:00010000:2.0:1789760796.071190:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.071205:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.071210:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.071216:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.071229:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.082126:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987200 is committed 00002000:00010000:3.0:1789760796.083108:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81481728 -> 81485824 00002000:00010000:3.0:1789760796.083115:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159168 -> 159176 00010000:00010000:3.0:1789760796.083130:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.083143:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.083152:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.083160:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.083165:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.083174:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.083194:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.083202:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f27fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81481728->81485823] (req 81481728->81485823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.083302:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936512:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12167us (12651us total) trans 4294987200 rc 0/0 00000100:00100000:3.0:1789760796.083313:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19904 00000100:00100000:3.0:1789760796.083323:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726182212400 time 726195724050 state 0x2 ref 0 00000100:00100000:0.0:1789760796.086338:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.087577:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936640 00000100:00100000:1.0:1789760796.087589:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936640 00000100:00100000:1.0:1789760796.087645:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726182212400 time 726195724050 state 0x0 ref 1 00000100:00100000:1.0:1789760796.087660:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726195984466 time 726195724050 state 0x1 ref 1 00000100:00100000:1.0:1789760796.087693:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726195984466 time 726212299182 state 0x0 ref 1 00000100:00100000:1.0:1789760796.087706:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503936640:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.087751:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81485824, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26dc8 00010000:00010000:1.0:1789760796.087760:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.087776:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.087780:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.087787:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.087801:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.094453:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987201 is committed 00002000:00010000:1.0:1789760796.099691:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81485824 -> 81489920 00002000:00010000:1.0:1789760796.099701:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159176 -> 159184 00010000:00010000:1.0:1789760796.099716:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.099731:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.099740:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.099748:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.099753:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.099763:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.100200:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.100210:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2803 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81485824->81489919] (req 81485824->81489919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.100312:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503936640:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12602us (13973us total) trans 4294987201 rc 0/0 00000100:00100000:1.0:1789760796.100321:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19905 00000100:00100000:1.0:1789760796.100332:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726195984466 time 726212299182 state 0x2 ref 0 00000100:00100000:1.0:1789760796.106681:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.106823:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936768 00000100:00100000:2.0:1789760796.106834:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936768 00000100:00100000:2.0:1789760796.106890:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726195984466 time 726212299182 state 0x0 ref 1 00000100:00100000:2.0:1789760796.106906:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726212559598 time 726212299182 state 0x1 ref 1 00000100:00100000:2.0:1789760796.106938:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726212559598 time 726231544395 state 0x0 ref 1 00000100:00100000:2.0:1789760796.106951:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.106997:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81489920, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26dd0 00010000:00010000:2.0:1789760796.107006:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.107022:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.107027:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.107033:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.107045:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.112880:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987202 is committed 00002000:00010000:1.0:1789760796.112989:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81489920 -> 81494016 00002000:00010000:1.0:1789760796.112994:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159184 -> 159192 00010000:00010000:1.0:1789760796.113041:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.113055:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.113064:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.113072:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.113076:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.113085:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.113103:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.113108:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f280a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81489920->81494015] (req 81489920->81494015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.113202:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503936768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6249us (6526us total) trans 4294987202 rc 0/0 00000100:00100000:1.0:1789760796.113213:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19906 00000100:00100000:1.0:1789760796.113223:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726212559598 time 726231544395 state 0x2 ref 0 00000100:00100000:3.0:1789760796.116422:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.116538:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503936896 00000100:00100000:1.0:1789760796.116549:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503936896 00000100:00100000:1.0:1789760796.116607:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726212559598 time 726231544395 state 0x0 ref 1 00000100:00100000:1.0:1789760796.116622:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726231804811 time 726231544395 state 0x1 ref 1 00000100:00100000:1.0:1789760796.116659:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726231804811 time 726241264888 state 0x0 ref 1 00000100:00100000:1.0:1789760796.116672:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503936896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.116722:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81494016, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26dd8 00010000:00010000:1.0:1789760796.116730:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.116743:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.116747:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.116753:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.116765:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.121643:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987203 is committed 00002000:00010000:1.0:1789760796.121825:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81494016 -> 81498112 00002000:00010000:1.0:1789760796.121830:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159192 -> 159200 00010000:00010000:1.0:1789760796.121843:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.121859:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.121866:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.121873:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.121878:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.121886:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.121905:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.121913:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2811 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81494016->81498111] (req 81494016->81498111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.122043:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8944a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503936896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5365us (5655us total) trans 4294987203 rc 0/0 00000100:00100000:1.0:1789760796.122053:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19907 00000100:00100000:1.0:1789760796.122071:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726231804811 time 726241264888 state 0x2 ref 0 00000100:00100000:0.0:1789760796.124195:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.124245:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937024 00000100:00100000:3.0:1789760796.124259:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937024 00000100:00100000:3.0:1789760796.124315:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726231804811 time 726241264888 state 0x0 ref 1 00000100:00100000:3.0:1789760796.124331:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726241525304 time 726241264888 state 0x1 ref 1 00000100:00100000:3.0:1789760796.124363:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726241525304 time 726248969428 state 0x0 ref 1 00000100:00100000:3.0:1789760796.124376:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.124451:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81498112, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26de0 00010000:00010000:3.0:1789760796.124472:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.124488:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.124492:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.124647:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.124665:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.133449:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987204 is committed 00002000:00010000:3.0:1789760796.136075:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81498112 -> 81502208 00002000:00010000:3.0:1789760796.136085:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159200 -> 159208 00010000:00010000:3.0:1789760796.136098:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.136111:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.136120:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.136126:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.136131:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.136140:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.136161:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.136170:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2818 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81498112->81502207] (req 81498112->81502207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.136271:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11893us (12077us total) trans 4294987204 rc 0/0 00000100:00100000:3.0:1789760796.136282:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19908 00000100:00100000:3.0:1789760796.136291:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726241525304 time 726248969428 state 0x2 ref 0 00000100:00100000:2.0:1789760796.141204:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.141294:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937152 00000100:00100000:1.0:1789760796.141304:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937152 00000100:00100000:1.0:1789760796.141360:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726241525304 time 726248969428 state 0x0 ref 1 00000100:00100000:1.0:1789760796.141376:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726249229844 time 726248969428 state 0x1 ref 1 00000100:00100000:1.0:1789760796.141589:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726249229844 time 726266194969 state 0x0 ref 1 00000100:00100000:1.0:1789760796.141603:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.141648:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81502208, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26de8 00010000:00010000:1.0:1789760796.141657:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.141672:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.141677:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.141683:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.141697:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.152734:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987205 is committed 00002000:00010000:1.0:1789760796.152857:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81502208 -> 81506304 00002000:00010000:1.0:1789760796.152862:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159208 -> 159216 00010000:00010000:1.0:1789760796.152876:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.152888:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.152902:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.152909:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.152914:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.152923:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.152944:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.152951:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f281f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81502208->81506303] (req 81502208->81506303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.153048:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11442us (11846us total) trans 4294987205 rc 0/0 00000100:00100000:1.0:1789760796.153057:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19909 00000100:00100000:1.0:1789760796.153065:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726249229844 time 726266194969 state 0x2 ref 0 00000100:00100000:1.0:1789760796.158598:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.158682:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937280 00000100:00100000:2.0:1789760796.158692:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937280 00000100:00100000:2.0:1789760796.158746:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726249229844 time 726266194969 state 0x0 ref 1 00000100:00100000:2.0:1789760796.158761:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726266455385 time 726266194969 state 0x1 ref 1 00000100:00100000:2.0:1789760796.158790:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726266455385 time 726283397754 state 0x0 ref 1 00000100:00100000:2.0:1789760796.158802:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.158848:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81506304, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26df0 00010000:00010000:2.0:1789760796.158856:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.158871:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.158875:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.158880:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.158893:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.166128:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987206 is committed 00002000:00010000:2.0:1789760796.166378:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81506304 -> 81510400 00002000:00010000:2.0:1789760796.166384:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159216 -> 159224 00010000:00010000:2.0:1789760796.166488:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.166502:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.166512:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.166519:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.166523:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.166532:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.166552:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.166560:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2826 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81506304->81510399] (req 81506304->81510399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.166807:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8000us (8213us total) trans 4294987206 rc 0/0 00000100:00100000:2.0:1789760796.166819:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19910 00000100:00100000:2.0:1789760796.166830:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726266455385 time 726283397754 state 0x2 ref 0 00000100:00100000:0.0:1789760796.173509:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.173634:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937408 00000100:00100000:3.0:1789760796.173643:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937408 00000100:00100000:3.0:1789760796.173695:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726266455385 time 726283397754 state 0x0 ref 1 00000100:00100000:3.0:1789760796.173710:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726283658170 time 726283397754 state 0x1 ref 1 00000100:00100000:3.0:1789760796.173741:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726283658170 time 726298347630 state 0x0 ref 1 00000100:00100000:3.0:1789760796.173754:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503937408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.173800:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81510400, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26df8 00010000:00010000:3.0:1789760796.173811:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.173827:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.173832:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.173838:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.173852:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.179829:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987207 is committed 00002000:00010000:3.0:1789760796.179948:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81510400 -> 81514496 00002000:00010000:3.0:1789760796.179953:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159224 -> 159232 00010000:00010000:3.0:1789760796.179965:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.179977:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.179986:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.179994:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.179998:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.180007:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.180027:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.180139:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f282d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81510400->81514495] (req 81510400->81514495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.180346:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503937408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6589us (6841us total) trans 4294987207 rc 0/0 00000100:00100000:3.0:1789760796.180357:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19911 00000100:00100000:3.0:1789760796.180366:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726283658170 time 726298347630 state 0x2 ref 0 00010000:00010000:1.0:1789760796.180537:0:6501:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760796.188131:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.188270:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937536 00000100:00100000:3.0:1789760796.188279:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937536 00000100:00100000:3.0:1789760796.188330:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726283658170 time 726298347630 state 0x0 ref 1 00000100:00100000:3.0:1789760796.188344:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726298608046 time 726298347630 state 0x1 ref 1 00000100:00100000:3.0:1789760796.188440:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726298608046 time 726313045574 state 0x0 ref 1 00000100:00100000:3.0:1789760796.188452:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503937536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.188522:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81514496, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e00 00010000:00010000:3.0:1789760796.188531:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.188547:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.188551:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.188557:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.188572:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.193228:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987208 is committed 00002000:00010000:2.0:1789760796.193924:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81514496 -> 81518592 00002000:00010000:2.0:1789760796.193931:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159232 -> 159240 00010000:00010000:2.0:1789760796.193943:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.193956:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.193965:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.193972:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.193977:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.193987:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.194006:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.194014:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2834 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81514496->81518591] (req 81514496->81518591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.194271:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503937536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5813us (6142us total) trans 4294987208 rc 0/0 00000100:00100000:2.0:1789760796.194283:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19912 00000100:00100000:2.0:1789760796.194292:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726298608046 time 726313045574 state 0x2 ref 0 00000100:00100000:2.0:1789760796.199686:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.199768:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937664 00000100:00100000:1.0:1789760796.199777:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937664 00000100:00100000:1.0:1789760796.199837:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726298608046 time 726313045574 state 0x0 ref 1 00000100:00100000:1.0:1789760796.199852:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726313305990 time 726313045574 state 0x1 ref 1 00000100:00100000:1.0:1789760796.199884:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726313305990 time 726324490604 state 0x0 ref 1 00000100:00100000:1.0:1789760796.199896:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503937664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.199941:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81518592, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e08 00010000:00010000:1.0:1789760796.199949:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.199964:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.199968:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.199974:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.199988:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.205857:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987209 is committed 00002000:00010000:1.0:1789760796.206084:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81518592 -> 81522688 00002000:00010000:1.0:1789760796.206090:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159240 -> 159248 00010000:00010000:1.0:1789760796.206104:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.206117:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.206132:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.206140:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.206145:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.206154:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.206174:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.206245:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f283b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81518592->81522687] (req 81518592->81522687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.206350:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503937664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6451us (6668us total) trans 4294987209 rc 0/0 00000100:00100000:1.0:1789760796.206359:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19913 00000100:00100000:1.0:1789760796.206369:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726313305990 time 726324490604 state 0x2 ref 0 00000100:00100000:0.0:1789760796.214489:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.214711:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937792 00000100:00100000:2.0:1789760796.214731:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937792 00000100:00100000:2.0:1789760796.214789:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726313305990 time 726324490604 state 0x0 ref 1 00000100:00100000:2.0:1789760796.214805:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726324751020 time 726324490604 state 0x1 ref 1 00000100:00100000:2.0:1789760796.214839:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726324751020 time 726339445270 state 0x0 ref 1 00000100:00100000:2.0:1789760796.214850:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.214897:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81522688, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e10 00010000:00010000:2.0:1789760796.214906:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.214922:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.214926:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.214932:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.214946:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.223675:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987210 is committed 00002000:00010000:2.0:1789760796.223900:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81522688 -> 81526784 00002000:00010000:2.0:1789760796.223905:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159248 -> 159256 00010000:00010000:2.0:1789760796.223920:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.223933:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.223941:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.223948:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.223953:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.223961:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.223981:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.223988:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2842 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81522688->81526783] (req 81522688->81526783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.224286:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503937792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9432us (9802us total) trans 4294987210 rc 0/0 00000100:00100000:2.0:1789760796.224300:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19914 00000100:00100000:2.0:1789760796.224310:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726324751020 time 726339445270 state 0x2 ref 0 00000100:00100000:2.0:1789760796.229257:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.229342:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503937920 00000100:00100000:3.0:1789760796.229352:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503937920 00000100:00100000:3.0:1789760796.229601:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726324751020 time 726339445270 state 0x0 ref 1 00000100:00100000:3.0:1789760796.229615:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726339705686 time 726339445270 state 0x1 ref 1 00000100:00100000:3.0:1789760796.229691:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726339705686 time 726354297248 state 0x0 ref 1 00000100:00100000:3.0:1789760796.229705:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503937920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.229766:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81526784, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e18 00010000:00010000:3.0:1789760796.229776:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.229791:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.229796:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.229802:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.229817:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.235371:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987211 is committed 00002000:00010000:3.0:1789760796.235690:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81526784 -> 81530880 00002000:00010000:3.0:1789760796.235695:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159256 -> 159264 00010000:00010000:3.0:1789760796.235708:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.235720:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.235735:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.235742:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.235747:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.235755:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.235773:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.235780:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2849 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81526784->81530879] (req 81526784->81530879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.235913:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503937920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6205us (6660us total) trans 4294987211 rc 0/0 00000100:00100000:3.0:1789760796.235925:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19915 00000100:00100000:3.0:1789760796.235934:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726339705686 time 726354297248 state 0x2 ref 0 00000100:00100000:1.0:1789760796.246239:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.247559:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938048 00000100:00100000:0.0:1789760796.247581:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938048 00000100:00100000:0.0:1789760796.247646:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726339705686 time 726354297248 state 0x0 ref 1 00000100:00100000:0.0:1789760796.247663:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726354557664 time 726354297248 state 0x1 ref 1 00000100:00100000:0.0:1789760796.247935:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726354557664 time 726372539093 state 0x0 ref 1 00000100:00100000:0.0:1789760796.247952:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503938048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.248007:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81530880, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e20 00010000:00010000:0.0:1789760796.248018:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.248034:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760796.248039:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.248045:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.248059:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.252974:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987212 is committed 00002000:00010000:0.0:1789760796.257239:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81530880 -> 81534976 00002000:00010000:0.0:1789760796.257250:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159264 -> 159272 00010000:00010000:0.0:1789760796.257264:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.257278:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.257293:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.257301:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760796.257306:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.257314:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.257383:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.257409:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2850 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81530880->81534975] (req 81530880->81534975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.257510:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503938048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9558us (11278us total) trans 4294987212 rc 0/0 00000100:00100000:0.0:1789760796.257519:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19916 00000100:00100000:0.0:1789760796.257529:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726354557664 time 726372539093 state 0x2 ref 0 00000100:00100000:0.0:1789760796.261877:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.261976:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938176 00000100:00100000:1.0:1789760796.261988:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938176 00000100:00100000:1.0:1789760796.262051:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726354557664 time 726372539093 state 0x0 ref 1 00000100:00100000:1.0:1789760796.262068:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726372799509 time 726372539093 state 0x1 ref 1 00000100:00100000:1.0:1789760796.262104:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726372799509 time 726386710247 state 0x0 ref 1 00000100:00100000:1.0:1789760796.262117:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503938176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.262290:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81534976, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e28 00010000:00010000:1.0:1789760796.262300:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.262316:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.262321:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.262327:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.262342:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.269088:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987213 is committed 00002000:00010000:1.0:1789760796.269264:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81534976 -> 81539072 00002000:00010000:1.0:1789760796.269270:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159272 -> 159280 00010000:00010000:1.0:1789760796.269288:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.269301:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.269312:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.269319:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.269324:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.269342:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.269362:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.269370:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f2857 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81534976->81539071] (req 81534976->81539071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.269525:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503938176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7405us (7654us total) trans 4294987213 rc 0/0 00000100:00100000:1.0:1789760796.269536:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19917 00000100:00100000:1.0:1789760796.269546:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726372799509 time 726386710247 state 0x2 ref 0 00000100:00100000:2.0:1789760796.277591:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.278008:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938304 00000100:00100000:1.0:1789760796.278019:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938304 00000100:00100000:1.0:1789760796.278073:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726372799509 time 726386710247 state 0x0 ref 1 00000100:00100000:1.0:1789760796.278088:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726386970663 time 726386710247 state 0x1 ref 1 00000100:00100000:1.0:1789760796.278743:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726386970663 time 726403346344 state 0x0 ref 1 00000100:00100000:1.0:1789760796.278761:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503938304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.278872:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81539072, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e30 00010000:00010000:1.0:1789760796.278881:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.278896:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.278900:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.278906:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.278925:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.284965:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987214 is committed 00002000:00010000:1.0:1789760796.285141:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81539072 -> 81543168 00002000:00010000:1.0:1789760796.285146:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159280 -> 159288 00010000:00010000:1.0:1789760796.285161:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.285174:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.285193:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.285205:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.285210:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.285219:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.285372:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.285381:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5b1ad04/0xfa2da65b083f285e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81539072->81543167] (req 81539072->81543167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.285656:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503938304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6895us (8070us total) trans 4294987214 rc 0/0 00000100:00100000:1.0:1789760796.285668:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19918 00000100:00100000:1.0:1789760796.285677:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726386970663 time 726403346344 state 0x2 ref 0 00000100:00100000:2.0:1789760796.289872:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.290347:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938432 00000100:00100000:3.0:1789760796.290358:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938432 00000100:00100000:3.0:1789760796.290449:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726386970663 time 726403346344 state 0x0 ref 1 00000100:00100000:3.0:1789760796.290634:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726403606760 time 726403346344 state 0x1 ref 1 00000100:00100000:3.0:1789760796.290666:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726403606760 time 726415272147 state 0x0 ref 1 00000100:00100000:3.0:1789760796.290679:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503938432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.290726:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81543168, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e38 00010000:00010000:3.0:1789760796.290736:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.290751:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.290756:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.290762:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.290776:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.301633:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987215 is committed 00002000:00010000:3.0:1789760796.301754:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81543168 -> 81547264 00002000:00010000:3.0:1789760796.301759:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159288 -> 159296 00010000:00010000:3.0:1789760796.301771:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301784:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301793:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301807:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.301812:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301822:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301842:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301850:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2865 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81543168->81547263] (req 81543168->81547263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.301943:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503938432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11262us (12076us total) trans 4294987215 rc 0/0 00000100:00100000:3.0:1789760796.301954:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19919 00000100:00100000:3.0:1789760796.301964:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726403606760 time 726415272147 state 0x2 ref 0 00000100:00100000:0.0:1789760796.304337:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.304619:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938560 00000100:00100000:2.0:1789760796.304629:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938560 00000100:00100000:2.0:1789760796.304680:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726403606760 time 726415272147 state 0x0 ref 1 00000100:00100000:2.0:1789760796.304695:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726415532563 time 726415272147 state 0x1 ref 1 00000100:00100000:2.0:1789760796.304727:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726415532563 time 726429333567 state 0x0 ref 1 00000100:00100000:2.0:1789760796.304739:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503938560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.304785:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81547264, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e40 00010000:00010000:2.0:1789760796.304793:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.304809:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.304814:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.304821:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.304840:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.311847:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987216 is committed 00002000:00010000:2.0:1789760796.311942:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81547264 -> 81551360 00002000:00010000:2.0:1789760796.311947:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159296 -> 159304 00010000:00010000:2.0:1789760796.311960:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.312863:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.312874:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.312882:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.312887:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.312897:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.312918:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.312926:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f286c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81547264->81551359] (req 81547264->81551359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.313074:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503938560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8333us (8741us total) trans 4294987216 rc 0/0 00000100:00100000:2.0:1789760796.313084:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19920 00000100:00100000:2.0:1789760796.313094:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726415532563 time 726429333567 state 0x2 ref 0 00000100:00100000:2.0:1789760796.318018:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.318291:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938688 00000100:00100000:1.0:1789760796.318302:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938688 00000100:00100000:1.0:1789760796.318358:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726415532563 time 726429333567 state 0x0 ref 1 00000100:00100000:1.0:1789760796.318374:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726429593983 time 726429333567 state 0x1 ref 1 00000100:00100000:1.0:1789760796.318420:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726429593983 time 726443026771 state 0x0 ref 1 00000100:00100000:1.0:1789760796.318436:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503938688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.319088:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81551360, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e48 00010000:00010000:1.0:1789760796.319099:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.319114:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.319119:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.319125:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.319138:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.327707:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987217 is committed 00002000:00010000:1.0:1789760796.327830:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81551360 -> 81555456 00002000:00010000:1.0:1789760796.327836:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159304 -> 159312 00010000:00010000:1.0:1789760796.327848:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.327862:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.327871:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.327878:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.327884:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.327893:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.327911:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.327919:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2873 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81551360->81555455] (req 81551360->81555455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.328010:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503938688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9572us (9998us total) trans 4294987217 rc 0/0 00000100:00100000:1.0:1789760796.328020:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19921 00000100:00100000:1.0:1789760796.328030:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726429593983 time 726443026771 state 0x2 ref 0 00000100:00100000:1.0:1789760796.335215:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.336685:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938816 00000100:00100000:2.0:1789760796.336701:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938816 00000100:00100000:2.0:1789760796.336763:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726429593983 time 726443026771 state 0x0 ref 1 00000100:00100000:2.0:1789760796.336778:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726443287187 time 726443026771 state 0x1 ref 1 00000100:00100000:2.0:1789760796.336815:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726443287187 time 726461421228 state 0x0 ref 1 00000100:00100000:2.0:1789760796.336828:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503938816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.336881:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81555456, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e50 00010000:00010000:2.0:1789760796.336891:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.336918:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.336923:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.336929:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.336943:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.345176:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987218 is committed 00002000:00010000:2.0:1789760796.348192:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81555456 -> 81559552 00002000:00010000:2.0:1789760796.348203:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159312 -> 159320 00010000:00010000:2.0:1789760796.348217:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.348230:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.348239:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.348245:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.348250:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.348259:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.348279:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.348286:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f287a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81555456->81559551] (req 81555456->81559551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.348477:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503938816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11645us (13265us total) trans 4294987218 rc 0/0 00000100:00100000:2.0:1789760796.348488:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19922 00000100:00100000:2.0:1789760796.348498:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726443287187 time 726461421228 state 0x2 ref 0 00000100:00100000:0.0:1789760796.354261:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.354497:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503938944 00000100:00100000:3.0:1789760796.354507:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503938944 00000100:00100000:3.0:1789760796.354570:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726443287187 time 726461421228 state 0x0 ref 1 00000100:00100000:3.0:1789760796.354585:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726461681644 time 726461421228 state 0x1 ref 1 00000100:00100000:3.0:1789760796.354620:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726461681644 time 726479226328 state 0x0 ref 1 00000100:00100000:3.0:1789760796.354631:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503938944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.354677:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81559552, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e58 00010000:00010000:3.0:1789760796.354686:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.354701:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.354706:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.354712:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.354725:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.359832:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987219 is committed 00002000:00010000:3.0:1789760796.360025:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81559552 -> 81563648 00002000:00010000:3.0:1789760796.360030:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159320 -> 159328 00010000:00010000:3.0:1789760796.360044:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.360057:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.360066:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.360079:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.360084:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.360092:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.360110:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.360117:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2881 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81559552->81563647] (req 81559552->81563647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.360479:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503938944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5826us (6204us total) trans 4294987219 rc 0/0 00000100:00100000:3.0:1789760796.360489:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19923 00000100:00100000:3.0:1789760796.360499:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726461681644 time 726479226328 state 0x2 ref 0 00000100:00100000:1.0:1789760796.363888:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.364044:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939072 00000100:00100000:0.0:1789760796.364055:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939072 00000100:00100000:0.0:1789760796.364108:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726461681644 time 726479226328 state 0x0 ref 1 00000100:00100000:0.0:1789760796.364123:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726479486744 time 726479226328 state 0x1 ref 1 00000100:00100000:0.0:1789760796.364154:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726479486744 time 726488761074 state 0x0 ref 1 00000100:00100000:0.0:1789760796.364167:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503939072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.364213:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81563648, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e60 00010000:00010000:0.0:1789760796.364223:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.364240:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760796.364244:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.364250:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.364264:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.369484:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987220 is committed 00002000:00010000:0.0:1789760796.369761:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81563648 -> 81567744 00002000:00010000:0.0:1789760796.369767:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159328 -> 159336 00010000:00010000:0.0:1789760796.369779:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.369791:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.369875:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.369883:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760796.369888:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.369897:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.369916:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.369924:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2888 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81563648->81567743] (req 81563648->81567743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.370054:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503939072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5884us (6171us total) trans 4294987220 rc 0/0 00000100:00100000:0.0:1789760796.370064:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19924 00000100:00100000:0.0:1789760796.370074:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726479486744 time 726488761074 state 0x2 ref 0 00000100:00100000:0.0:1789760796.373044:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.373128:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939200 00000100:00100000:1.0:1789760796.373138:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939200 00000100:00100000:1.0:1789760796.373210:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726479486744 time 726488761074 state 0x0 ref 1 00000100:00100000:1.0:1789760796.373225:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726489021490 time 726488761074 state 0x1 ref 1 00000100:00100000:1.0:1789760796.373250:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726489021490 time 726497856234 state 0x0 ref 1 00000100:00100000:1.0:1789760796.373263:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.373306:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81567744, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e68 00010000:00010000:1.0:1789760796.373315:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.373330:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.373335:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.373341:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.373360:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.378262:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987221 is committed 00002000:00010000:1.0:1789760796.378485:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81567744 -> 81571840 00002000:00010000:1.0:1789760796.378491:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159336 -> 159344 00010000:00010000:1.0:1789760796.378504:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378517:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378526:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378533:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.378538:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378547:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378566:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378574:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f288f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81567744->81571839] (req 81567744->81571839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.378671:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5406us (5630us total) trans 4294987221 rc 0/0 00000100:00100000:1.0:1789760796.378681:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19925 00000100:00100000:1.0:1789760796.378690:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726489021490 time 726497856234 state 0x2 ref 0 00000100:00100000:0.0:1789760796.388311:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.388701:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939328 00000100:00100000:2.0:1789760796.388712:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939328 00000100:00100000:2.0:1789760796.388766:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726489021490 time 726497856234 state 0x0 ref 1 00000100:00100000:2.0:1789760796.388781:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726498116650 time 726497856234 state 0x1 ref 1 00000100:00100000:2.0:1789760796.388811:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726498116650 time 726513417638 state 0x0 ref 1 00000100:00100000:2.0:1789760796.388833:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.388879:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81571840, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e70 00010000:00010000:2.0:1789760796.388888:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.389324:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.389330:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.389337:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.389353:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.395822:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987222 is committed 00002000:00010000:2.0:1789760796.395939:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81571840 -> 81575936 00002000:00010000:2.0:1789760796.395944:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159344 -> 159352 00010000:00010000:2.0:1789760796.395957:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.395969:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.395977:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.395984:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.395988:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.395997:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.396017:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.396023:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f2896 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81571840->81575935] (req 81571840->81575935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.396112:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7277us (7808us total) trans 4294987222 rc 0/0 00000100:00100000:2.0:1789760796.396130:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19926 00000100:00100000:2.0:1789760796.396140:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726498116650 time 726513417638 state 0x2 ref 0 00000100:00100000:2.0:1789760796.398728:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.398827:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939456 00000100:00100000:3.0:1789760796.398837:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939456 00000100:00100000:3.0:1789760796.398893:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726498116650 time 726513417638 state 0x0 ref 1 00000100:00100000:3.0:1789760796.398909:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726513678054 time 726513417638 state 0x1 ref 1 00000100:00100000:3.0:1789760796.398979:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726513678054 time 726523585392 state 0x0 ref 1 00000100:00100000:3.0:1789760796.398995:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.399039:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81575936, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e78 00010000:00010000:3.0:1789760796.399049:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.399065:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.399069:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.399076:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.399090:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.408196:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987223 is committed 00002000:00010000:3.0:1789760796.408302:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81575936 -> 81580032 00002000:00010000:3.0:1789760796.408308:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159352 -> 159360 00010000:00010000:3.0:1789760796.408322:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.408335:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.408344:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.408350:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.408355:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.408364:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.408384:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.408419:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f289d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81575936->81580031] (req 81575936->81580031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.408514:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9515us (9790us total) trans 4294987223 rc 0/0 00000100:00100000:3.0:1789760796.408524:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19927 00000100:00100000:3.0:1789760796.408533:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726513678054 time 726523585392 state 0x2 ref 0 00000100:00100000:0.0:1789760796.413627:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.413720:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939584 00000100:00100000:2.0:1789760796.413730:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939584 00000100:00100000:2.0:1789760796.413794:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726513678054 time 726523585392 state 0x0 ref 1 00000100:00100000:2.0:1789760796.413809:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726523845808 time 726523585392 state 0x1 ref 1 00000100:00100000:2.0:1789760796.413853:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726523845808 time 726538458842 state 0x0 ref 1 00000100:00100000:2.0:1789760796.413866:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503939584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.413917:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81580032, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e80 00010000:00010000:2.0:1789760796.413927:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.413943:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.413948:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.413954:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.413969:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.419433:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987224 is committed 00002000:00010000:2.0:1789760796.420909:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81580032 -> 81584128 00002000:00010000:2.0:1789760796.420916:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159360 -> 159368 00010000:00010000:2.0:1789760796.420929:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.420949:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.420957:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.421027:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.421033:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.421043:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.421065:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.421073:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81580032->81584127] (req 81580032->81584127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.421207:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503939584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7338us (7584us total) trans 4294987224 rc 0/0 00000100:00100000:2.0:1789760796.421219:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19928 00000100:00100000:2.0:1789760796.421229:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726523845808 time 726538458842 state 0x2 ref 0 00000100:00100000:0.0:1789760796.427897:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.428562:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939712 00000100:00100000:1.0:1789760796.428574:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939712 00000100:00100000:1.0:1789760796.428636:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726523845808 time 726538458842 state 0x0 ref 1 00000100:00100000:1.0:1789760796.428650:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726538719258 time 726538458842 state 0x1 ref 1 00000100:00100000:1.0:1789760796.428721:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726538719258 time 726553327301 state 0x0 ref 1 00000100:00100000:1.0:1789760796.428733:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503939712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.428781:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81584128, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e88 00010000:00010000:1.0:1789760796.428853:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.428871:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.428876:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.428882:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.428895:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.437898:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987225 is committed 00002000:00010000:2.0:1789760796.438034:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81584128 -> 81588224 00002000:00010000:2.0:1789760796.438039:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159368 -> 159376 00010000:00010000:2.0:1789760796.438053:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.438067:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.438076:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.438083:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.438087:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.438096:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.438114:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.438121:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f28ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81584128->81588223] (req 81584128->81588223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.438223:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503939712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9487us (10332us total) trans 4294987225 rc 0/0 00000100:00100000:2.0:1789760796.438234:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19929 00000100:00100000:2.0:1789760796.438244:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726538719258 time 726553327301 state 0x2 ref 0 00000100:00100000:1.0:1789760796.443228:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.443438:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939840 00000100:00100000:2.0:1789760796.444059:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939840 00000100:00100000:2.0:1789760796.444119:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726538719258 time 726553327301 state 0x0 ref 1 00000100:00100000:2.0:1789760796.444146:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726553587717 time 726553327301 state 0x1 ref 1 00000100:00100000:2.0:1789760796.444181:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726553587717 time 726568786981 state 0x0 ref 1 00000100:00100000:2.0:1789760796.444193:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.444362:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81588224, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e90 00010000:00010000:2.0:1789760796.444371:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.444387:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.444542:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.444550:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.444570:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.451190:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987226 is committed 00002000:00010000:2.0:1789760796.451339:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81588224 -> 81592320 00002000:00010000:2.0:1789760796.451344:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159376 -> 159384 00010000:00010000:2.0:1789760796.451357:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.451376:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.451385:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.451524:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.451529:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.451539:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.451559:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.451566:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81588224->81592319] (req 81588224->81592319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.451690:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7494us (8467us total) trans 4294987226 rc 0/0 00000100:00100000:2.0:1789760796.451701:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19930 00000100:00100000:2.0:1789760796.451710:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726553587717 time 726568786981 state 0x2 ref 0 00000100:00100000:0.0:1789760796.456691:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.456780:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503939968 00000100:00100000:3.0:1789760796.456790:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503939968 00000100:00100000:3.0:1789760796.456843:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726553587717 time 726568786981 state 0x0 ref 1 00000100:00100000:3.0:1789760796.456857:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726569047397 time 726568786981 state 0x1 ref 1 00000100:00100000:3.0:1789760796.456885:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726569047397 time 726581491751 state 0x0 ref 1 00000100:00100000:3.0:1789760796.456897:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.456939:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81592320, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26e98 00010000:00010000:3.0:1789760796.456949:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.456965:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.456969:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.456974:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.456987:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.462163:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987227 is committed 00002000:00010000:3.0:1789760796.462624:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81592320 -> 81596416 00002000:00010000:3.0:1789760796.462631:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159384 -> 159392 00010000:00010000:3.0:1789760796.462645:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.462658:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.462667:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.462675:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.462680:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.462689:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.462708:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.462716:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81592320->81596415] (req 81592320->81596415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.462805:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503939968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5904us (6118us total) trans 4294987227 rc 0/0 00000100:00100000:3.0:1789760796.462816:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19931 00000100:00100000:3.0:1789760796.462827:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726569047397 time 726581491751 state 0x2 ref 0 00000100:00100000:0.0:1789760796.470550:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.471708:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940096 00000100:00100000:2.0:1789760796.471721:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940096 00000100:00100000:2.0:1789760796.471781:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726569047397 time 726581491751 state 0x0 ref 1 00000100:00100000:2.0:1789760796.471795:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726581752167 time 726581491751 state 0x1 ref 1 00000100:00100000:2.0:1789760796.471831:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726581752167 time 726596437070 state 0x0 ref 1 00000100:00100000:2.0:1789760796.471844:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503940096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.471893:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81596416, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ea0 00010000:00010000:2.0:1789760796.471901:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.471918:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.471923:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.471929:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.471943:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.475711:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987228 is committed 00002000:00010000:2.0:1789760796.475901:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81596416 -> 81600512 00002000:00010000:2.0:1789760796.475907:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159392 -> 159400 00010000:00010000:2.0:1789760796.475920:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.475933:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.475949:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.475957:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.475961:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.475971:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.475989:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.475996:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098da032a/0xfa2da65b083f28c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81596416->81600511] (req 81596416->81600511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.476114:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503940096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4268us (5933us total) trans 4294987228 rc 0/0 00000100:00100000:2.0:1789760796.476125:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19932 00000100:00100000:2.0:1789760796.476135:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726581752167 time 726596437070 state 0x2 ref 0 00000100:00100000:1.0:1789760796.479257:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.479354:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940224 00000100:00100000:2.0:1789760796.479364:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940224 00000100:00100000:2.0:1789760796.479497:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726581752167 time 726596437070 state 0x0 ref 1 00000100:00100000:2.0:1789760796.479520:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726596697486 time 726596437070 state 0x1 ref 1 00000100:00100000:2.0:1789760796.479584:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726596697486 time 726604190143 state 0x0 ref 1 00000100:00100000:2.0:1789760796.479596:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503940224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.479640:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81600512, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ea8 00010000:00010000:2.0:1789760796.479648:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.479663:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.479667:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.479673:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.479686:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.483278:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987229 is committed 00002000:00010000:2.0:1789760796.483448:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81600512 -> 81604608 00002000:00010000:2.0:1789760796.483471:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159400 -> 159408 00010000:00010000:2.0:1789760796.483485:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.483497:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.483506:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.483519:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.483524:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.483533:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.483552:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.483598:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f28c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81600512->81604607] (req 81600512->81604607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.483679:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503940224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4081us (4429us total) trans 4294987229 rc 0/0 00000100:00100000:2.0:1789760796.483686:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19933 00000100:00100000:2.0:1789760796.483695:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726596697486 time 726604190143 state 0x2 ref 0 00000100:00100000:0.0:1789760796.487110:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.487213:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940352 00000100:00100000:1.0:1789760796.487222:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940352 00000100:00100000:1.0:1789760796.487277:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726596697486 time 726604190143 state 0x0 ref 1 00000100:00100000:1.0:1789760796.487293:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726604450559 time 726604190143 state 0x1 ref 1 00000100:00100000:1.0:1789760796.487326:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726604450559 time 726611932421 state 0x0 ref 1 00000100:00100000:1.0:1789760796.487339:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503940352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.487387:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81604608, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26eb0 00010000:00010000:1.0:1789760796.487426:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.487440:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.487445:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.487452:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.487483:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.491640:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987230 is committed 00002000:00010000:1.0:1789760796.492031:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81604608 -> 81608704 00002000:00010000:1.0:1789760796.492036:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159408 -> 159416 00010000:00010000:1.0:1789760796.492049:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.492061:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.492070:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.492077:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.492082:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.492091:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.492111:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.492119:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f28ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81604608->81608703] (req 81604608->81608703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.492218:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503940352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4876us (5113us total) trans 4294987230 rc 0/0 00000100:00100000:1.0:1789760796.492227:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19934 00000100:00100000:1.0:1789760796.492237:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726604450559 time 726611932421 state 0x2 ref 0 00000100:00100000:0.0:1789760796.500227:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.500370:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940480 00000100:00100000:3.0:1789760796.500416:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940480 00000100:00100000:3.0:1789760796.500491:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726604450559 time 726611932421 state 0x0 ref 1 00000100:00100000:3.0:1789760796.500790:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726612192837 time 726611932421 state 0x1 ref 1 00000100:00100000:3.0:1789760796.500824:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726612192837 time 726625430556 state 0x0 ref 1 00000100:00100000:3.0:1789760796.500838:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff527b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503940480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.500883:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81608704, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26eb8 00010000:00010000:3.0:1789760796.500892:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.500907:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.500912:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.500918:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.500932:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.506520:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987231 is committed 00002000:00010000:3.0:1789760796.506671:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81608704 -> 81612800 00002000:00010000:3.0:1789760796.506676:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159416 -> 159424 00010000:00010000:3.0:1789760796.506752:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.506766:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.506775:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.506783:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.506788:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.506797:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.506825:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.506837:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81608704->81612799] (req 81608704->81612799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.507023:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff527b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503940480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6184us (6802us total) trans 4294987231 rc 0/0 00000100:00100000:3.0:1789760796.507033:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19935 00000100:00100000:3.0:1789760796.507043:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726612192837 time 726625430556 state 0x2 ref 0 00000100:00100000:2.0:1789760796.511617:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.511815:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940608 00000100:00100000:0.0:1789760796.511825:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940608 00000100:00100000:0.0:1789760796.511880:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726612192837 time 726625430556 state 0x0 ref 1 00000100:00100000:0.0:1789760796.511894:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726625690972 time 726625430556 state 0x1 ref 1 00000100:00100000:0.0:1789760796.511930:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726625690972 time 726636536281 state 0x0 ref 1 00000100:00100000:0.0:1789760796.511943:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503940608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.511989:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81612800, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ec0 00010000:00010000:0.0:1789760796.511998:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.512012:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760796.512016:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.512022:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.512035:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.519097:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987232 is committed 00002000:00010000:0.0:1789760796.519311:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81612800 -> 81616896 00002000:00010000:0.0:1789760796.519317:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159424 -> 159432 00010000:00010000:0.0:1789760796.519330:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.519342:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.519351:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.519358:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760796.519363:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.519372:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.519752:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.519763:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f28dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81612800->81616895] (req 81612800->81616895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.519895:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503940608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7950us (8282us total) trans 4294987232 rc 0/0 00000100:00100000:0.0:1789760796.519906:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19936 00000100:00100000:0.0:1789760796.519916:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726625690972 time 726636536281 state 0x2 ref 0 00000100:00100000:0.0:1789760796.524296:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.524706:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940736 00000100:00100000:2.0:1789760796.524717:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940736 00000100:00100000:2.0:1789760796.524778:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726625690972 time 726636536281 state 0x0 ref 1 00000100:00100000:2.0:1789760796.524794:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726636796697 time 726636536281 state 0x1 ref 1 00000100:00100000:2.0:1789760796.524863:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726636796697 time 726649468744 state 0x0 ref 1 00000100:00100000:2.0:1789760796.524876:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503940736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.524923:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81616896, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ec8 00010000:00010000:2.0:1789760796.524932:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.524948:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.524952:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.524958:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.524972:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.529837:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987233 is committed 00002000:00010000:2.0:1789760796.530266:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81616896 -> 81620992 00002000:00010000:2.0:1789760796.530272:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159432 -> 159440 00010000:00010000:2.0:1789760796.530285:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.530297:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.530306:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.530318:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.530323:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.530332:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.530351:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.530358:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f28e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81616896->81620991] (req 81616896->81620991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.530755:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503940736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5874us (6463us total) trans 4294987233 rc 0/0 00000100:00100000:2.0:1789760796.530766:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19937 00000100:00100000:2.0:1789760796.530775:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726636796697 time 726649468744 state 0x2 ref 0 00000100:00100000:2.0:1789760796.541682:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.541895:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940864 00000100:00100000:1.0:1789760796.541907:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940864 00000100:00100000:1.0:1789760796.541965:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726636796697 time 726649468744 state 0x0 ref 1 00000100:00100000:1.0:1789760796.541981:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726649729160 time 726649468744 state 0x1 ref 1 00000100:00100000:1.0:1789760796.542009:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726649729160 time 726666615838 state 0x0 ref 1 00000100:00100000:1.0:1789760796.542021:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503940864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.542069:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81620992, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ed0 00010000:00010000:1.0:1789760796.542078:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.542101:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.542105:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.542111:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.542123:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.547988:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987234 is committed 00002000:00010000:2.0:1789760796.548099:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81620992 -> 81625088 00002000:00010000:2.0:1789760796.548104:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159440 -> 159448 00010000:00010000:2.0:1789760796.548117:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.548130:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.548140:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.548147:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.548151:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.548160:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.548179:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.548198:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f28ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81620992->81625087] (req 81620992->81625087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.548290:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503940864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6266us (6613us total) trans 4294987234 rc 0/0 00000100:00100000:2.0:1789760796.548300:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19938 00000100:00100000:2.0:1789760796.548309:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726649729160 time 726666615838 state 0x2 ref 0 00000100:00100000:1.0:1789760796.560245:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.561484:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503940992 00000100:00100000:3.0:1789760796.561497:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503940992 00000100:00100000:3.0:1789760796.561551:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726649729160 time 726666615838 state 0x0 ref 1 00000100:00100000:3.0:1789760796.561576:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726666876254 time 726666615838 state 0x1 ref 1 00000100:00100000:3.0:1789760796.561609:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726666876254 time 726686215857 state 0x0 ref 1 00000100:00100000:3.0:1789760796.561622:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503940992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.561670:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81625088, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ed8 00010000:00010000:3.0:1789760796.561680:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.561695:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.561700:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.561707:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.561729:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.567948:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987235 is committed 00002000:00010000:3.0:1789760796.568069:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81625088 -> 81629184 00002000:00010000:3.0:1789760796.568074:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159448 -> 159456 00010000:00010000:3.0:1789760796.568087:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.568100:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.568108:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.568115:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.568119:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.568127:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.568147:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.568154:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f28f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81625088->81629183] (req 81625088->81629183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.568345:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503940992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6712us (8095us total) trans 4294987235 rc 0/0 00000100:00100000:3.0:1789760796.568356:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19939 00000100:00100000:3.0:1789760796.568367:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726666876254 time 726686215857 state 0x2 ref 0 00000100:00100000:0.0:1789760796.576772:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.576890:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503941120 00000100:00100000:2.0:1789760796.576900:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503941120 00000100:00100000:2.0:1789760796.576955:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726666876254 time 726686215857 state 0x0 ref 1 00000100:00100000:2.0:1789760796.576972:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726686476273 time 726686215857 state 0x1 ref 1 00000100:00100000:2.0:1789760796.577050:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726686476273 time 726701656746 state 0x0 ref 1 00000100:00100000:2.0:1789760796.577063:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.577115:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81629184, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ee0 00010000:00010000:2.0:1789760796.577124:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.577139:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.577143:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.577149:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.577161:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.584919:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987236 is committed 00002000:00010000:2.0:1789760796.585109:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81629184 -> 81633280 00002000:00010000:2.0:1789760796.585115:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159456 -> 159464 00010000:00010000:2.0:1789760796.585129:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.585141:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.585150:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.585162:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.585167:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.585176:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.585313:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.585320:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f28f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81629184->81633279] (req 81629184->81633279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.585684:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8616us (8916us total) trans 4294987236 rc 0/0 00000100:00100000:2.0:1789760796.585696:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19940 00000100:00100000:2.0:1789760796.585706:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726686476273 time 726701656746 state 0x2 ref 0 00000100:00100000:2.0:1789760796.589384:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.590079:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503941248 00000100:00100000:1.0:1789760796.590091:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503941248 00000100:00100000:1.0:1789760796.590149:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726686476273 time 726701656746 state 0x0 ref 1 00000100:00100000:1.0:1789760796.590164:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726701917162 time 726701656746 state 0x1 ref 1 00000100:00100000:1.0:1789760796.590199:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726701917162 time 726714805729 state 0x0 ref 1 00000100:00100000:1.0:1789760796.590212:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503941248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.590265:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81633280, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ee8 00010000:00010000:1.0:1789760796.590275:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.590291:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.590296:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.590302:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.590317:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.601504:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987237 is committed 00002000:00010000:2.0:1789760796.602798:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81633280 -> 81637376 00002000:00010000:2.0:1789760796.602806:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159464 -> 159472 00010000:00010000:2.0:1789760796.602822:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.602836:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.602845:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.602852:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.602857:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.602866:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.602886:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.602955:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f28ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81633280->81637375] (req 81633280->81637375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.603441:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503941248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13171us (14007us total) trans 4294987237 rc 0/0 00000100:00100000:2.0:1789760796.603454:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19941 00000100:00100000:2.0:1789760796.603475:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726701917162 time 726714805729 state 0x2 ref 0 00000100:00100000:1.0:1789760796.613381:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.615090:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503941376 00000100:00100000:2.0:1789760796.615104:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503941376 00000100:00100000:2.0:1789760796.615165:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726701917162 time 726714805729 state 0x0 ref 1 00000100:00100000:2.0:1789760796.615615:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726715066145 time 726714805729 state 0x1 ref 1 00000100:00100000:2.0:1789760796.615648:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726715066145 time 726740254267 state 0x0 ref 1 00000100:00100000:2.0:1789760796.615664:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.615850:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81637376, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ef0 00010000:00010000:2.0:1789760796.615861:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.615876:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.615880:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.615887:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.615901:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.632925:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987238 is committed 00002000:00010000:0.0:1789760796.633091:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81637376 -> 81641472 00002000:00010000:0.0:1789760796.633096:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159472 -> 159480 00010000:00010000:0.0:1789760796.633108:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.633121:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.633130:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.633138:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760796.633143:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.633151:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.633172:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.633196:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000397ac64f/0xfa2da65b083f2906 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81637376->81641471] (req 81637376->81641471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.633292:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 17627us (19914us total) trans 4294987238 rc 0/0 00000100:00100000:0.0:1789760796.633303:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19942 00000100:00100000:0.0:1789760796.633314:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726715066145 time 726740254267 state 0x2 ref 0 00000100:00100000:0.0:1789760796.641619:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.641860:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503941504 00000100:00100000:3.0:1789760796.641870:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503941504 00000100:00100000:3.0:1789760796.641925:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726715066145 time 726740254267 state 0x0 ref 1 00000100:00100000:3.0:1789760796.641942:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726740514683 time 726740254267 state 0x1 ref 1 00000100:00100000:3.0:1789760796.642040:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726740514683 time 726766645891 state 0x0 ref 1 00000100:00100000:3.0:1789760796.642054:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503941504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.642104:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81641472, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26ef8 00010000:00010000:3.0:1789760796.642113:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.642129:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.642133:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.642140:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.642153:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.654362:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987239 is committed 00002000:00010000:3.0:1789760796.654631:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81641472 -> 81645568 00002000:00010000:3.0:1789760796.654636:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159480 -> 159488 00010000:00010000:3.0:1789760796.654649:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.654660:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.654668:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.654674:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.654679:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.654687:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.654704:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.654711:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f290d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81641472->81645567] (req 81641472->81645567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.655158:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503941504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13100us (13659us total) trans 4294987239 rc 0/0 00000100:00100000:3.0:1789760796.655168:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19943 00000100:00100000:3.0:1789760796.655176:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726740514683 time 726766645891 state 0x2 ref 0 00000100:00100000:2.0:1789760796.668194:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.668284:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503941632 00000100:00100000:0.0:1789760796.668294:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503941632 00000100:00100000:0.0:1789760796.668348:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726740514683 time 726766645891 state 0x0 ref 1 00000100:00100000:0.0:1789760796.668362:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726766906307 time 726766645891 state 0x1 ref 1 00000100:00100000:0.0:1789760796.668430:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726766906307 time 726793036138 state 0x0 ref 1 00000100:00100000:0.0:1789760796.668443:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.668511:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81645568, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26f00 00010000:00010000:0.0:1789760796.668520:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.668535:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760796.668539:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.668546:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.668560:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.681142:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987240 is committed 00002000:00010000:0.0:1789760796.681349:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81645568 -> 81649664 00002000:00010000:0.0:1789760796.681354:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159488 -> 159496 00010000:00010000:0.0:1789760796.681367:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.681380:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.681408:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.681416:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760796.681421:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.681431:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.681449:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.681472:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2914 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81645568->81649663] (req 81645568->81649663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.681632:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13187us (13452us total) trans 4294987240 rc 0/0 00000100:00100000:0.0:1789760796.681643:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19944 00000100:00100000:0.0:1789760796.681652:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726766906307 time 726793036138 state 0x2 ref 0 00000100:00100000:1.0:1789760796.685518:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.685671:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503941760 00000100:00100000:2.0:1789760796.685682:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503941760 00000100:00100000:2.0:1789760796.685749:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726766906307 time 726793036138 state 0x0 ref 1 00000100:00100000:2.0:1789760796.685765:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726793296554 time 726793036138 state 0x1 ref 1 00000100:00100000:2.0:1789760796.685789:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726793296554 time 726810395819 state 0x0 ref 1 00000100:00100000:2.0:1789760796.685802:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503941760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.685851:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81649664, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26f08 00010000:00010000:2.0:1789760796.685861:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.685877:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.685881:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.685887:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.685900:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.698890:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987241 is committed 00002000:00010000:2.0:1789760796.700281:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81649664 -> 81653760 00002000:00010000:2.0:1789760796.700289:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159496 -> 159504 00010000:00010000:2.0:1789760796.700302:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.700314:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.700322:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.700329:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.700334:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.700348:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.700366:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.700374:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f291b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81649664->81653759] (req 81649664->81653759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.700494:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503941760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14689us (14981us total) trans 4294987241 rc 0/0 00000100:00100000:2.0:1789760796.700504:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19945 00000100:00100000:2.0:1789760796.700514:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726793296554 time 726810395819 state 0x2 ref 0 00000100:00100000:2.0:1789760796.704484:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.704689:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503941888 00000100:00100000:1.0:1789760796.704699:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503941888 00000100:00100000:1.0:1789760796.704823:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726793296554 time 726810395819 state 0x0 ref 1 00000100:00100000:1.0:1789760796.704839:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726810656235 time 726810395819 state 0x1 ref 1 00000100:00100000:1.0:1789760796.704902:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726810656235 time 726829508116 state 0x0 ref 1 00000100:00100000:1.0:1789760796.704915:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.704974:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81653760, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26f10 00010000:00010000:1.0:1789760796.704984:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.705000:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.705005:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.705013:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.705027:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.712077:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987242 is committed 00002000:00010000:1.0:1789760796.712378:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81653760 -> 81657856 00002000:00010000:1.0:1789760796.712383:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159504 -> 159512 00010000:00010000:1.0:1789760796.712432:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.712445:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.712453:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.712470:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.712476:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.712485:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.712896:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.712912:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e6cf63d/0xfa2da65b083f2922 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81653760->81657855] (req 81653760->81657855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.713018:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503941888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8100us (8635us total) trans 4294987242 rc 0/0 00000100:00100000:1.0:1789760796.713381:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19946 00000100:00100000:1.0:1789760796.713427:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726810656235 time 726829508116 state 0x2 ref 0 00000100:00100000:3.0:1789760796.721034:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.721193:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942016 00000100:00100000:1.0:1789760796.721203:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942016 00000100:00100000:1.0:1789760796.721258:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726810656235 time 726829508116 state 0x0 ref 1 00000100:00100000:1.0:1789760796.721272:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726829768532 time 726829508116 state 0x1 ref 1 00000100:00100000:1.0:1789760796.721305:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726829768532 time 726845911441 state 0x0 ref 1 00000100:00100000:1.0:1789760796.721317:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.721366:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81657856, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26f18 00010000:00010000:1.0:1789760796.721375:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.721418:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.721424:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.721430:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.721450:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.734436:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987243 is committed 00002000:00010000:3.0:1789760796.735736:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81657856 -> 81661952 00002000:00010000:3.0:1789760796.735745:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159512 -> 159520 00010000:00010000:3.0:1789760796.735758:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.735771:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.735779:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.735787:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760796.735792:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.735801:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.735820:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.735834:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2929 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81657856->81661951] (req 81657856->81661951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.735942:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8947480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14621us (14913us total) trans 4294987243 rc 0/0 00000100:00100000:3.0:1789760796.735954:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19947 00000100:00100000:3.0:1789760796.735975:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726829768532 time 726845911441 state 0x2 ref 0 00000100:00100000:3.0:1789760796.743169:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.744084:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942144 00000100:00100000:0.0:1789760796.744096:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942144 00000100:00100000:0.0:1789760796.744156:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726829768532 time 726845911441 state 0x0 ref 1 00000100:00100000:0.0:1789760796.744173:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726846171857 time 726845911441 state 0x1 ref 1 00000100:00100000:0.0:1789760796.744679:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726846171857 time 726869282398 state 0x0 ref 1 00000100:00100000:0.0:1789760796.744695:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.744744:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81661952, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26f20 00010000:00010000:0.0:1789760796.744754:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.744769:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760796.744774:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.744780:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.744798:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.754864:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987244 is committed 00002000:00010000:2.0:1789760796.755155:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81661952 -> 81666048 00002000:00010000:2.0:1789760796.755160:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159520 -> 159528 00010000:00010000:2.0:1789760796.755174:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.755187:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.755197:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.755210:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.755215:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.755225:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.755246:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.755253:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2930 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81661952->81666047] (req 81661952->81666047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.755417:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10691us (12221us total) trans 4294987244 rc 0/0 00000100:00100000:2.0:1789760796.755429:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19948 00000100:00100000:2.0:1789760796.755439:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726846171857 time 726869282398 state 0x2 ref 0 00000100:00100000:1.0:1789760796.768797:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.768910:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942272 00000100:00100000:2.0:1789760796.768920:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942272 00000100:00100000:2.0:1789760796.768979:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726846171857 time 726869282398 state 0x0 ref 1 00000100:00100000:2.0:1789760796.768996:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726869542814 time 726869282398 state 0x1 ref 1 00000100:00100000:2.0:1789760796.769026:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726869542814 time 726893633121 state 0x0 ref 1 00000100:00100000:2.0:1789760796.769041:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.769091:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81666048, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26f28 00010000:00010000:2.0:1789760796.769101:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.769117:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.769121:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.769127:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.769140:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.784068:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987245 is committed 00002000:00010000:2.0:1789760796.784269:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81666048 -> 81670144 00002000:00010000:2.0:1789760796.784275:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159528 -> 159536 00010000:00010000:2.0:1789760796.784287:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.784300:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.784308:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.784315:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.784321:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.784329:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.784348:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.784355:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2937 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81666048->81670143] (req 81666048->81670143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.784528:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15482us (15736us total) trans 4294987245 rc 0/0 00000100:00100000:2.0:1789760796.784539:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19949 00000100:00100000:2.0:1789760796.784549:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726869542814 time 726893633121 state 0x2 ref 0 00000100:00100000:0.0:1789760796.791758:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.799161:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942400 00000100:00100000:1.0:1789760796.799187:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942400 00000100:00100000:1.0:1789760796.799257:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726869542814 time 726893633121 state 0x0 ref 1 00000100:00100000:1.0:1789760796.799274:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726893893537 time 726893633121 state 0x1 ref 1 00000100:00100000:1.0:1789760796.799359:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726893893537 time 726923964246 state 0x0 ref 1 00000100:00100000:1.0:1789760796.799374:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.799442:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81670144, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x26f30 00010000:00010000:1.0:1789760796.799452:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.799479:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.799484:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.799490:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.799505:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.805594:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987246 is committed 00002000:00010000:1.0:1789760796.805829:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81670144 -> 81674240 00002000:00010000:1.0:1789760796.805834:0:8434:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:1.0:1789760796.805837:0:8434:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00002000:00010000:1.0:1789760796.805839:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159536 -> 159544 00010000:00010000:1.0:1789760796.805852:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.805865:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.805875:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.805883:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.805888:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.805898:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.805919:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.805927:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3138574/0xfa2da65b083f293e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81670144->81674239] (req 81670144->81674239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.806029:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6653us (14276us total) trans 4294987246 rc 0/0 00000100:00100000:1.0:1789760796.806039:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19950 00000100:00100000:1.0:1789760796.806049:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726893893537 time 726923964246 state 0x2 ref 0 00000100:00100000:1.0:1789760796.813233:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.816261:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942528 00000100:00100000:3.0:1789760796.816279:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942528 00000100:00100000:3.0:1789760796.816355:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726893893537 time 726923964246 state 0x0 ref 1 00000100:00100000:3.0:1789760796.816374:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726924224662 time 726923964246 state 0x1 ref 1 00000100:00100000:3.0:1789760796.816466:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726924224662 time 726941061543 state 0x0 ref 1 00000100:00100000:3.0:1789760796.816481:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.816544:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81674240, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f38 00010000:00010000:3.0:1789760796.816554:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.817037:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760796.817042:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.817049:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.817065:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.823922:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987247 is committed 00002000:00010000:1.0:1789760796.830859:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81674240 -> 81678336 00002000:00010000:1.0:1789760796.830870:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159544 -> 159552 00010000:00010000:1.0:1789760796.830884:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.830898:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.830906:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.830913:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.830918:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.830926:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.830945:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.830953:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2945 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81674240->81678335] (req 81674240->81678335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.831104:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14621us (17875us total) trans 4294987247 rc 0/0 00000100:00100000:1.0:1789760796.831115:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19951 00000100:00100000:1.0:1789760796.831126:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726924224662 time 726941061543 state 0x2 ref 0 00000100:00100000:0.0:1789760796.840332:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.840473:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942656 00000100:00100000:2.0:1789760796.840484:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942656 00000100:00100000:2.0:1789760796.840548:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726924224662 time 726941061543 state 0x0 ref 1 00000100:00100000:2.0:1789760796.840576:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726941321959 time 726941061543 state 0x1 ref 1 00000100:00100000:2.0:1789760796.840650:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726941321959 time 726965256528 state 0x0 ref 1 00000100:00100000:2.0:1789760796.840664:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.840711:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81678336, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f40 00010000:00010000:2.0:1789760796.840720:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.840736:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760796.840740:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.840746:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.840762:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.854253:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987248 is committed 00002000:00010000:2.0:1789760796.856262:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81678336 -> 81682432 00002000:00010000:2.0:1789760796.856272:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159552 -> 159560 00010000:00010000:2.0:1789760796.856287:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856300:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856309:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856317:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.856322:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856331:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856353:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856360:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f294c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81678336->81682431] (req 81678336->81682431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.856834:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16167us (16506us total) trans 4294987248 rc 0/0 00000100:00100000:2.0:1789760796.856846:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19952 00000100:00100000:2.0:1789760796.856856:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726941321959 time 726965256528 state 0x2 ref 0 00000100:00100000:2.0:1789760796.861230:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.861528:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942784 00000100:00100000:1.0:1789760796.861538:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942784 00000100:00100000:1.0:1789760796.861601:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726941321959 time 726965256528 state 0x0 ref 1 00000100:00100000:1.0:1789760796.861616:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726965516944 time 726965256528 state 0x1 ref 1 00000100:00100000:1.0:1789760796.861646:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726965516944 time 726986252879 state 0x0 ref 1 00000100:00100000:1.0:1789760796.861660:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.861706:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81682432, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f48 00010000:00010000:1.0:1789760796.861715:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.861730:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760796.861734:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.861741:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.861760:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.873444:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987249 is committed 00002000:00010000:1.0:1789760796.873812:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81682432 -> 81686528 00002000:00010000:1.0:1789760796.873817:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159560 -> 159568 00010000:00010000:1.0:1789760796.873831:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.873845:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.873854:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.873861:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.873866:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.873875:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.873898:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.873905:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2953 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81682432->81686527] (req 81682432->81686527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.874053:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503942784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12391us (12829us total) trans 4294987249 rc 0/0 00000100:00100000:1.0:1789760796.874064:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19953 00000100:00100000:1.0:1789760796.874074:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726965516944 time 726986252879 state 0x2 ref 0 00000100:00100000:2.0:1789760796.883801:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5dff564380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523617152:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760796.883874:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760796.884038:0:7941:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523617152 00000100:00100000:1.0:1789760796.884046:0:7941:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523617152 00000100:00100000:1.0:1789760796.884098:0:7941:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1789760796.884107:0:7941:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523617152:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760796.884141:0:7941:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:1.0:1789760796.884197:0:7941:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3638:x1876699523617152:12345-0@lo:41:osp-pre-1-0.0 Request processed in 91us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760796.884204:0:7941:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:0.0:1789760796.884336:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5dff564380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3638:1876699523617152:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760796.885954:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.886031:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503942912 00000100:00100000:0.0:1789760796.886039:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503942912 00000100:00100000:0.0:1789760796.886086:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726965516944 time 726986252879 state 0x0 ref 1 00000100:00100000:0.0:1789760796.886099:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726986513295 time 726986252879 state 0x1 ref 1 00000100:00100000:0.0:1789760796.886126:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726986513295 time 727010733847 state 0x0 ref 1 00000100:00100000:0.0:1789760796.886137:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.886181:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81686528, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f50 00010000:00010000:0.0:1789760796.886190:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.886205:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760796.886209:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.886215:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.886232:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.894031:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987250 is committed 00002000:00010000:0.0:1789760796.894148:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81686528 -> 81690624 00002000:00010000:0.0:1789760796.894153:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159568 -> 159576 00010000:00010000:0.0:1789760796.894166:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.894232:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.894241:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.894249:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760796.894253:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.894261:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.894282:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.894289:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f295a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81686528->81690623] (req 81686528->81690623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.894734:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503942912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8591us (8778us total) trans 4294987250 rc 0/0 00000100:00100000:0.0:1789760796.894745:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19954 00000100:00100000:0.0:1789760796.894755:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726986513295 time 727010733847 state 0x2 ref 0 00000100:00100000:0.0:1789760796.898259:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760796.901404:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943040 00000100:00100000:3.0:1789760796.901422:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943040 00000100:00100000:3.0:1789760796.901623:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 726986513295 time 727010733847 state 0x0 ref 1 00000100:00100000:3.0:1789760796.901640:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727010994263 time 727010733847 state 0x1 ref 1 00000100:00100000:3.0:1789760796.901687:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727010994263 time 727026293641 state 0x0 ref 1 00000100:00100000:3.0:1789760796.901701:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff564380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503943040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760796.901758:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81690624, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f58 00010000:00010000:3.0:1789760796.901767:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.901784:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760796.901788:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.901794:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.901809:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.909104:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987251 is committed 00002000:00010000:1.0:1789760796.912732:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81690624 -> 81694720 00002000:00010000:1.0:1789760796.912742:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159576 -> 159584 00010000:00010000:1.0:1789760796.912756:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.912768:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.912777:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.912789:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.912794:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.912803:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.912822:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.912829:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a32edf0e/0xfa2da65b083f2961 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81690624->81694719] (req 81690624->81694719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.912933:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff564380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503943040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11230us (14679us total) trans 4294987251 rc 0/0 00000100:00100000:1.0:1789760796.912944:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19955 00000100:00100000:1.0:1789760796.912954:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727010994263 time 727026293641 state 0x2 ref 0 00000100:00100000:2.0:1789760796.916332:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.916429:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943168 00000100:00100000:0.0:1789760796.916439:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943168 00000100:00100000:0.0:1789760796.916513:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727010994263 time 727026293641 state 0x0 ref 1 00000100:00100000:0.0:1789760796.916528:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727026554057 time 727026293641 state 0x1 ref 1 00000100:00100000:0.0:1789760796.916549:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727026554057 time 727041155999 state 0x0 ref 1 00000100:00100000:0.0:1789760796.916560:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503943168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.916608:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81694720, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f60 00010000:00010000:0.0:1789760796.916618:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.916634:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760796.916639:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.916645:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.916664:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.924413:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987252 is committed 00002000:00010000:0.0:1789760796.924531:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81694720 -> 81698816 00002000:00010000:0.0:1789760796.924536:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159584 -> 159592 00010000:00010000:0.0:1789760796.924549:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.924569:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.924579:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.924586:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760796.924590:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.924599:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.924619:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.924626:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2968 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81694720->81698815] (req 81694720->81698815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.924720:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503943168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8157us (8392us total) trans 4294987252 rc 0/0 00000100:00100000:0.0:1789760796.924730:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19956 00000100:00100000:0.0:1789760796.924740:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727026554057 time 727041155999 state 0x2 ref 0 00000100:00100000:3.0:1789760796.927991:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.928085:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943296 00000100:00100000:1.0:1789760796.928094:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943296 00000100:00100000:1.0:1789760796.928151:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727026554057 time 727041155999 state 0x0 ref 1 00000100:00100000:1.0:1789760796.928168:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727041416415 time 727041155999 state 0x1 ref 1 00000100:00100000:1.0:1789760796.928293:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727041416415 time 727052898677 state 0x0 ref 1 00000100:00100000:1.0:1789760796.928308:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503943296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.928356:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81698816, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f68 00010000:00010000:1.0:1789760796.928364:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.928378:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760796.928382:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.928413:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.928427:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.934654:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987253 is committed 00002000:00010000:1.0:1789760796.934864:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81698816 -> 81702912 00002000:00010000:1.0:1789760796.934869:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159592 -> 159600 00010000:00010000:1.0:1789760796.934881:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934893:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934902:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934909:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.934915:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934924:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934942:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.934950:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f296f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81698816->81702911] (req 81698816->81702911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.935088:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8944380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503943296:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6777us (7595us total) trans 4294987253 rc 0/0 00000100:00100000:1.0:1789760796.935099:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19957 00000100:00100000:1.0:1789760796.935108:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727041416415 time 727052898677 state 0x2 ref 0 00000100:00100000:0.0:1789760796.941910:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.942006:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943424 00000100:00100000:2.0:1789760796.942016:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943424 00000100:00100000:2.0:1789760796.942068:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727041416415 time 727052898677 state 0x0 ref 1 00000100:00100000:2.0:1789760796.942083:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727053159093 time 727052898677 state 0x1 ref 1 00000100:00100000:2.0:1789760796.942143:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727053159093 time 727066749485 state 0x0 ref 1 00000100:00100000:2.0:1789760796.942156:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503943424:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.942219:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81702912, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f70 00010000:00010000:2.0:1789760796.942229:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.942246:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760796.942250:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.942257:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.942271:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.947439:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987254 is committed 00002000:00010000:2.0:1789760796.948897:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81702912 -> 81707008 00002000:00010000:2.0:1789760796.948904:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159600 -> 159608 00010000:00010000:2.0:1789760796.948919:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.948931:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.948940:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.948948:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.948954:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.948963:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.948983:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.948991:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2976 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81702912->81707007] (req 81702912->81707007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.949196:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503943424:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7037us (7292us total) trans 4294987254 rc 0/0 00000100:00100000:2.0:1789760796.949207:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19958 00000100:00100000:2.0:1789760796.949217:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727053159093 time 727066749485 state 0x2 ref 0 00000100:00100000:2.0:1789760796.955350:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760796.955533:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943552 00000100:00100000:1.0:1789760796.955546:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943552 00000100:00100000:1.0:1789760796.955639:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727053159093 time 727066749485 state 0x0 ref 1 00000100:00100000:1.0:1789760796.955657:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727067009901 time 727066749485 state 0x1 ref 1 00000100:00100000:1.0:1789760796.955693:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727067009901 time 727080300029 state 0x0 ref 1 00000100:00100000:1.0:1789760796.955706:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc10c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503943552:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760796.955768:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81707008, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f78 00010000:00010000:1.0:1789760796.955779:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.955801:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760796.955805:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.955813:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.955829:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.964353:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987255 is committed 00002000:00010000:1.0:1789760796.966954:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81707008 -> 81711104 00002000:00010000:1.0:1789760796.966970:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159608 -> 159616 00010000:00010000:1.0:1789760796.966993:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.967009:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.967020:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.967029:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760796.967035:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.967045:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.967073:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.967081:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f297d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81707008->81711103] (req 81707008->81711103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.967204:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc10c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503943552:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11492us (11864us total) trans 4294987255 rc 0/0 00000100:00100000:1.0:1789760796.967215:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19959 00000100:00100000:1.0:1789760796.967228:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727067009901 time 727080300029 state 0x2 ref 0 00000100:00100000:2.0:1789760796.973849:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760796.974086:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943680 00000100:00100000:0.0:1789760796.974100:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943680 00000100:00100000:0.0:1789760796.974195:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727067009901 time 727080300029 state 0x0 ref 1 00000100:00100000:0.0:1789760796.974217:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727080560445 time 727080300029 state 0x1 ref 1 00000100:00100000:0.0:1789760796.974250:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727080560445 time 727098856564 state 0x0 ref 1 00000100:00100000:0.0:1789760796.974265:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503943680:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760796.974336:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81711104, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f80 00010000:00010000:0.0:1789760796.974348:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.974368:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760796.974374:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.974382:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.974462:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.980775:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987256 is committed 00002000:00010000:2.0:1789760796.981798:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81711104 -> 81715200 00002000:00010000:2.0:1789760796.981808:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159616 -> 159624 00010000:00010000:2.0:1789760796.981827:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.981842:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.981852:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.981861:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.981868:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.981878:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.981902:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.981910:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2984 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81711104->81715199] (req 81711104->81715199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.982049:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503943680:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7779us (8209us total) trans 4294987256 rc 0/0 00000100:00100000:2.0:1789760796.982061:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19960 00000100:00100000:2.0:1789760796.982073:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727080560445 time 727098856564 state 0x2 ref 0 00000100:00100000:0.0:1789760796.992570:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760796.992717:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943808 00000100:00100000:2.0:1789760796.992732:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943808 00000100:00100000:2.0:1789760796.992793:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727080560445 time 727098856564 state 0x0 ref 1 00000100:00100000:2.0:1789760796.992807:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727099116980 time 727098856564 state 0x1 ref 1 00000100:00100000:2.0:1789760796.992840:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727099116980 time 727117447159 state 0x0 ref 1 00000100:00100000:2.0:1789760796.992852:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503943808:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760796.992901:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81715200, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f88 00010000:00010000:2.0:1789760796.992911:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.992927:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760796.992931:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.992937:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.992950:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.998567:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987257 is committed 00002000:00010000:2.0:1789760796.998782:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81715200 -> 81719296 00002000:00010000:2.0:1789760796.998789:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159624 -> 159632 00010000:00010000:2.0:1789760796.998807:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.998821:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.998831:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.998838:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760796.998843:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.998852:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.998877:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.998885:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f298b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81715200->81719295] (req 81715200->81719295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.999117:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503943808:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6259us (6555us total) trans 4294987257 rc 0/0 00000100:00100000:2.0:1789760796.999130:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19961 00000100:00100000:2.0:1789760796.999142:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727099116980 time 727117447159 state 0x2 ref 0 00000100:00100000:0.0:1789760797.005684:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.005824:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503943936 00000100:00100000:2.0:1789760797.005838:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503943936 00000100:00100000:2.0:1789760797.005922:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727099116980 time 727117447159 state 0x0 ref 1 00000100:00100000:2.0:1789760797.005943:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727117707575 time 727117447159 state 0x1 ref 1 00000100:00100000:2.0:1789760797.005982:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727117707575 time 727130587781 state 0x0 ref 1 00000100:00100000:2.0:1789760797.005996:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503943936:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.006061:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81719296, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f90 00010000:00010000:2.0:1789760797.006073:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.006092:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.006098:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.006106:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.006123:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.016731:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987258 is committed 00002000:00010000:2.0:1789760797.018686:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81719296 -> 81723392 00002000:00010000:2.0:1789760797.018703:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159632 -> 159640 00010000:00010000:2.0:1789760797.018722:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.018737:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.018746:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.018754:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.018759:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.018770:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.018794:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.018808:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2992 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81719296->81723391] (req 81719296->81723391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.019349:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503943936:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13346us (13671us total) trans 4294987258 rc 0/0 00000100:00100000:2.0:1789760797.019364:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19962 00000100:00100000:2.0:1789760797.019377:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727117707575 time 727130587781 state 0x2 ref 0 00000100:00100000:1.0:1789760797.027055:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.027250:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944064 00000100:00100000:2.0:1789760797.027264:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944064 00000100:00100000:2.0:1789760797.027357:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727117707575 time 727130587781 state 0x0 ref 1 00000100:00100000:2.0:1789760797.027376:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727130848197 time 727130587781 state 0x1 ref 1 00000100:00100000:2.0:1789760797.027434:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727130848197 time 727152040185 state 0x0 ref 1 00000100:00100000:2.0:1789760797.027448:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944064:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.027744:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81723392, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26f98 00010000:00010000:2.0:1789760797.027754:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.027774:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.027779:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.027786:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.027809:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.036377:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987259 is committed 00002000:00010000:2.0:1789760797.036751:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81723392 -> 81727488 00002000:00010000:2.0:1789760797.036756:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159640 -> 159648 00010000:00010000:2.0:1789760797.036774:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.036789:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.036800:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.036808:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.036814:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.036824:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.036852:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.036860:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f2999 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81723392->81727487] (req 81723392->81727487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.036987:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944064:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9533us (9940us total) trans 4294987259 rc 0/0 00000100:00100000:2.0:1789760797.037000:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19963 00000100:00100000:2.0:1789760797.037013:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727130848197 time 727152040185 state 0x2 ref 0 00000100:00100000:0.0:1789760797.043910:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.044017:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944192 00000100:00100000:2.0:1789760797.044038:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944192 00000100:00100000:2.0:1789760797.044111:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727130848197 time 727152040185 state 0x0 ref 1 00000100:00100000:2.0:1789760797.044129:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727152300601 time 727152040185 state 0x1 ref 1 00000100:00100000:2.0:1789760797.044164:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727152300601 time 727168770891 state 0x0 ref 1 00000100:00100000:2.0:1789760797.044462:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503944192:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.044657:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81727488, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fa0 00010000:00010000:2.0:1789760797.044670:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.044689:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.044693:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.044700:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.044716:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.054946:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987260 is committed 00002000:00010000:2.0:1789760797.055341:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81727488 -> 81731584 00002000:00010000:2.0:1789760797.055348:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159648 -> 159656 00010000:00010000:2.0:1789760797.055365:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.055380:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.056132:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.056149:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.056154:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.056167:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.056271:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.056281:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81727488->81731583] (req 81727488->81731583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.056764:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503944192:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12580us (12857us total) trans 4294987260 rc 0/0 00000100:00100000:2.0:1789760797.056779:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19964 00000100:00100000:2.0:1789760797.056793:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727152300601 time 727168770891 state 0x2 ref 0 00000100:00100000:0.0:1789760797.063002:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.063095:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944320 00000100:00100000:1.0:1789760797.063109:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944320 00000100:00100000:1.0:1789760797.063187:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727152300601 time 727168770891 state 0x0 ref 1 00000100:00100000:1.0:1789760797.063206:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727169031307 time 727168770891 state 0x1 ref 1 00000100:00100000:1.0:1789760797.063300:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727169031307 time 727187904614 state 0x0 ref 1 00000100:00100000:1.0:1789760797.063316:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944320:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.063383:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81731584, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fa8 00010000:00010000:1.0:1789760797.063438:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.063474:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.063479:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.063487:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.063515:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.070618:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987261 is committed 00002000:00010000:1.0:1789760797.070869:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81731584 -> 81735680 00002000:00010000:1.0:1789760797.070876:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159656 -> 159664 00010000:00010000:1.0:1789760797.070892:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.070907:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.070917:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.070926:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.070932:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.070942:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.070971:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.070979:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a488ea74/0xfa2da65b083f29a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81731584->81735679] (req 81731584->81735679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.071112:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944320:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7790us (8117us total) trans 4294987261 rc 0/0 00000100:00100000:1.0:1789760797.071124:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19965 00000100:00100000:1.0:1789760797.071137:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727169031307 time 727187904614 state 0x2 ref 0 00000100:00100000:1.0:1789760797.076785:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.076916:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944448 00000100:00100000:2.0:1789760797.076930:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944448 00000100:00100000:2.0:1789760797.077004:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727169031307 time 727187904614 state 0x0 ref 1 00000100:00100000:2.0:1789760797.077101:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727188165030 time 727187904614 state 0x1 ref 1 00000100:00100000:2.0:1789760797.077136:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727188165030 time 727201742589 state 0x0 ref 1 00000100:00100000:2.0:1789760797.077151:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503944448:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.077382:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81735680, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fb0 00010000:00010000:2.0:1789760797.077483:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.077631:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.077636:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.077644:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.077662:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.084785:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987262 is committed 00002000:00010000:0.0:1789760797.085336:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81735680 -> 81739776 00002000:00010000:0.0:1789760797.085343:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159664 -> 159672 00010000:00010000:0.0:1789760797.085361:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.085377:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.085386:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.085626:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.085632:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.085643:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.085669:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.085683:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000893ee297/0xfa2da65b083f29ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81735680->81739775] (req 81735680->81739775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.086111:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503944448:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8953us (9332us total) trans 4294987262 rc 0/0 00000100:00100000:0.0:1789760797.086124:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19966 00000100:00100000:0.0:1789760797.086137:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727188165030 time 727201742589 state 0x2 ref 0 00000100:00100000:1.0:1789760797.097361:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.097666:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944576 00000100:00100000:2.0:1789760797.097680:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944576 00000100:00100000:2.0:1789760797.097757:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727188165030 time 727201742589 state 0x0 ref 1 00000100:00100000:2.0:1789760797.097843:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727202003005 time 727201742589 state 0x1 ref 1 00000100:00100000:2.0:1789760797.097875:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727202003005 time 727222480805 state 0x0 ref 1 00000100:00100000:2.0:1789760797.097890:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944576:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.097955:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81739776, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fb8 00010000:00010000:2.0:1789760797.097967:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.097987:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.097992:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.097999:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.098015:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.106716:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987263 is committed 00002000:00010000:2.0:1789760797.106981:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81739776 -> 81743872 00002000:00010000:2.0:1789760797.106988:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159672 -> 159680 00010000:00010000:2.0:1789760797.107005:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.107021:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.107037:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.107046:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.107051:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.107061:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.107086:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.107094:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f29b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81739776->81743871] (req 81739776->81743871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.107479:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944576:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9584us (10126us total) trans 4294987263 rc 0/0 00000100:00100000:2.0:1789760797.107499:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19967 00000100:00100000:2.0:1789760797.107513:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727202003005 time 727222480805 state 0x2 ref 0 00000100:00100000:2.0:1789760797.114159:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.114606:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944704 00000100:00100000:0.0:1789760797.114620:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944704 00000100:00100000:0.0:1789760797.114699:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727202003005 time 727222480805 state 0x0 ref 1 00000100:00100000:0.0:1789760797.114719:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727222741221 time 727222480805 state 0x1 ref 1 00000100:00100000:0.0:1789760797.114761:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727222741221 time 727239366800 state 0x0 ref 1 00000100:00100000:0.0:1789760797.114783:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503944704:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.114849:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81743872, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fc0 00010000:00010000:0.0:1789760797.114860:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.114881:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.114887:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.114895:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.114916:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.120050:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987264 is committed 00002000:00010000:0.0:1789760797.120280:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81743872 -> 81747968 00002000:00010000:0.0:1789760797.120286:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159680 -> 159688 00010000:00010000:0.0:1789760797.120305:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.120319:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.120329:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.120336:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.120342:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.120352:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.120378:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.120386:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81743872->81747967] (req 81743872->81747967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.120698:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503944704:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5911us (6547us total) trans 4294987264 rc 0/0 00000100:00100000:0.0:1789760797.120711:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19968 00000100:00100000:0.0:1789760797.120725:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727222741221 time 727239366800 state 0x2 ref 0 00000100:00100000:2.0:1789760797.130742:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.130947:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944832 00000100:00100000:1.0:1789760797.130961:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944832 00000100:00100000:1.0:1789760797.131035:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727222741221 time 727239366800 state 0x0 ref 1 00000100:00100000:1.0:1789760797.131054:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727239627216 time 727239366800 state 0x1 ref 1 00000100:00100000:1.0:1789760797.131122:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727239627216 time 727255727921 state 0x0 ref 1 00000100:00100000:1.0:1789760797.131137:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944832:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.131217:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81747968, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fc8 00010000:00010000:1.0:1789760797.131229:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.131250:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.131256:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.131264:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.131281:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.141032:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987265 is committed 00002000:00010000:1.0:1789760797.141198:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81747968 -> 81752064 00002000:00010000:1.0:1789760797.141204:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159688 -> 159696 00010000:00010000:1.0:1789760797.141222:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.141236:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.141246:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.141254:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.141259:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.141276:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.141299:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.141320:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81747968->81752063] (req 81747968->81752063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.141487:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944832:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10344us (10754us total) trans 4294987265 rc 0/0 00000100:00100000:1.0:1789760797.141499:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19969 00000100:00100000:1.0:1789760797.141512:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727239627216 time 727255727921 state 0x2 ref 0 00000100:00100000:1.0:1789760797.152162:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.152448:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503944960 00000100:00100000:0.0:1789760797.152471:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503944960 00000100:00100000:0.0:1789760797.152563:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727239627216 time 727255727921 state 0x0 ref 1 00000100:00100000:0.0:1789760797.152583:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727255988337 time 727255727921 state 0x1 ref 1 00000100:00100000:0.0:1789760797.152622:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727255988337 time 727277228050 state 0x0 ref 1 00000100:00100000:0.0:1789760797.152636:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944960:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.152701:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81752064, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fd0 00010000:00010000:0.0:1789760797.152714:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.152735:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.152740:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.152748:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.152764:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.159945:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987266 is committed 00002000:00010000:0.0:1789760797.160080:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81752064 -> 81756160 00002000:00010000:0.0:1789760797.160088:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159696 -> 159704 00010000:00010000:0.0:1789760797.160106:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.160121:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.160131:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.160138:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.160144:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.160154:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.160178:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.160247:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81752064->81756159] (req 81752064->81756159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.160871:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503944960:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8227us (8714us total) trans 4294987266 rc 0/0 00000100:00100000:0.0:1789760797.160886:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19970 00000100:00100000:0.0:1789760797.160898:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727255988337 time 727277228050 state 0x2 ref 0 00000100:00100000:2.0:1789760797.164431:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.164680:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945088 00000100:00100000:1.0:1789760797.164693:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945088 00000100:00100000:1.0:1789760797.164874:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727255988337 time 727277228050 state 0x0 ref 1 00000100:00100000:1.0:1789760797.164896:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727277488466 time 727277228050 state 0x1 ref 1 00000100:00100000:1.0:1789760797.164932:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727277488466 time 727289538991 state 0x0 ref 1 00000100:00100000:1.0:1789760797.164947:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503945088:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.165012:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81756160, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fd8 00010000:00010000:1.0:1789760797.165023:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165043:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.165048:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.165057:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165074:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.171364:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987267 is committed 00002000:00010000:1.0:1789760797.171565:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81756160 -> 81760256 00002000:00010000:1.0:1789760797.171572:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159704 -> 159712 00010000:00010000:1.0:1789760797.171590:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.171607:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.171617:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.171626:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.171632:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.171649:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.171679:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.171692:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81756160->81760255] (req 81756160->81760255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.172004:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503945088:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7051us (7622us total) trans 4294987267 rc 0/0 00000100:00100000:1.0:1789760797.172020:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19971 00000100:00100000:1.0:1789760797.172035:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727277488466 time 727289538991 state 0x2 ref 0 00000100:00100000:2.0:1789760797.178207:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.178371:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945216 00000100:00100000:0.0:1789760797.178382:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945216 00000100:00100000:0.0:1789760797.178497:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727277488466 time 727289538991 state 0x0 ref 1 00000100:00100000:0.0:1789760797.178516:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727289799407 time 727289538991 state 0x1 ref 1 00000100:00100000:0.0:1789760797.178542:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727289799407 time 727303148553 state 0x0 ref 1 00000100:00100000:0.0:1789760797.178556:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503945216:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.178748:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81760256, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fe0 00010000:00010000:0.0:1789760797.178760:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.178777:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.178781:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.178786:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.178800:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.184183:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987268 is committed 00002000:00010000:0.0:1789760797.184327:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81760256 -> 81764352 00002000:00010000:0.0:1789760797.184333:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159712 -> 159720 00010000:00010000:0.0:1789760797.184350:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.184361:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.184368:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.184372:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.184377:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.184385:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.184493:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.184502:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f29d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81760256->81764351] (req 81760256->81764351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.184901:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503945216:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6338us (6698us total) trans 4294987268 rc 0/0 00000100:00100000:1.0:1789760797.184961:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19972 00000100:00100000:1.0:1789760797.184977:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727289799407 time 727303148553 state 0x2 ref 0 00000100:00100000:2.0:1789760797.193760:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.193923:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945344 00000100:00100000:1.0:1789760797.193938:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945344 00000100:00100000:1.0:1789760797.194020:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727289799407 time 727303148553 state 0x0 ref 1 00000100:00100000:1.0:1789760797.194040:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727303408969 time 727303148553 state 0x1 ref 1 00000100:00100000:1.0:1789760797.194081:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727303408969 time 727318687245 state 0x0 ref 1 00000100:00100000:1.0:1789760797.194096:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503945344:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.194165:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81764352, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26fe8 00010000:00010000:1.0:1789760797.194178:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.194199:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.194205:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.194213:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.194236:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.204299:0:7454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760797.205787:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987269 is committed 00002000:00010000:2.0:1789760797.205961:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81764352 -> 81768448 00002000:00010000:2.0:1789760797.205968:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159720 -> 159728 00010000:00010000:2.0:1789760797.205985:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.206000:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.206010:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.206018:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.206023:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.206035:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.206062:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.206072:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f29df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81764352->81768447] (req 81764352->81768447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.206298:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503945344:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12196us (12546us total) trans 4294987269 rc 0/0 00000100:00100000:2.0:1789760797.206313:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19973 00000100:00100000:2.0:1789760797.206325:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727303408969 time 727318687245 state 0x2 ref 0 00000100:00100000:2.0:1789760797.217565:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.217796:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945472 00000100:00100000:1.0:1789760797.217810:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945472 00000100:00100000:1.0:1789760797.217890:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727303408969 time 727318687245 state 0x0 ref 1 00000100:00100000:1.0:1789760797.217910:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727318947661 time 727318687245 state 0x1 ref 1 00000100:00100000:1.0:1789760797.217952:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727318947661 time 727342556775 state 0x0 ref 1 00000100:00100000:1.0:1789760797.217967:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503945472:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.218039:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81768448, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26ff0 00010000:00010000:1.0:1789760797.218051:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.218071:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.218077:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.218084:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.218102:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.224466:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987270 is committed 00002000:00010000:0.0:1789760797.225305:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81768448 -> 81772544 00002000:00010000:0.0:1789760797.225316:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159728 -> 159736 00010000:00010000:0.0:1789760797.225335:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225685:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225696:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225704:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.225839:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225852:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225879:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225888:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f29e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81768448->81772543] (req 81768448->81772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.226025:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503945472:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8054us (8470us total) trans 4294987270 rc 0/0 00000100:00100000:0.0:1789760797.226037:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19974 00000100:00100000:0.0:1789760797.226049:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727318947661 time 727342556775 state 0x2 ref 0 00000100:00100000:1.0:1789760797.235241:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.235510:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945600 00000100:00100000:2.0:1789760797.235524:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945600 00000100:00100000:2.0:1789760797.235602:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727318947661 time 727342556775 state 0x0 ref 1 00000100:00100000:2.0:1789760797.235623:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727342817191 time 727342556775 state 0x1 ref 1 00000100:00100000:2.0:1789760797.235661:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727342817191 time 727360266700 state 0x0 ref 1 00000100:00100000:2.0:1789760797.235676:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503945600:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.235743:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81772544, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x26ff8 00010000:00010000:2.0:1789760797.235755:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235777:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.235782:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.235790:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.235809:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.250984:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987271 is committed 00002000:00010000:2.0:1789760797.251339:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81772544 -> 81776640 00002000:00010000:2.0:1789760797.251347:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159736 -> 159744 00010000:00010000:2.0:1789760797.251364:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.251382:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.251438:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.251447:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.251461:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.251472:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.251618:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.251628:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f29ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81772544->81776639] (req 81772544->81776639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.251761:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503945600:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16080us (16527us total) trans 4294987271 rc 0/0 00000100:00100000:2.0:1789760797.252312:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19975 00000100:00100000:2.0:1789760797.252326:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727342817191 time 727360266700 state 0x2 ref 0 00000100:00100000:2.0:1789760797.261325:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.261555:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945728 00000100:00100000:1.0:1789760797.261568:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945728 00000100:00100000:1.0:1789760797.261643:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727342817191 time 727360266700 state 0x0 ref 1 00000100:00100000:1.0:1789760797.261664:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727360527116 time 727360266700 state 0x1 ref 1 00000100:00100000:1.0:1789760797.261703:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727360527116 time 727386309296 state 0x0 ref 1 00000100:00100000:1.0:1789760797.261718:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503945728:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.261857:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81776640, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27000 00010000:00010000:1.0:1789760797.261870:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.261892:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.261897:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.261905:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.261922:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.270063:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987272 is committed 00002000:00010000:0.0:1789760797.270207:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81776640 -> 81780736 00002000:00010000:0.0:1789760797.270215:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159744 -> 159752 00010000:00010000:0.0:1789760797.270232:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.270247:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.270257:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.270266:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.270281:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.270292:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.270317:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.270326:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f29f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81776640->81780735] (req 81776640->81780735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.270796:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503945728:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9072us (9476us total) trans 4294987272 rc 0/0 00000100:00100000:0.0:1789760797.270810:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19976 00000100:00100000:0.0:1789760797.270823:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727360527116 time 727386309296 state 0x2 ref 0 00000100:00100000:0.0:1789760797.276876:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.277533:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945856 00000100:00100000:1.0:1789760797.277547:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945856 00000100:00100000:1.0:1789760797.277627:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727360527116 time 727386309296 state 0x0 ref 1 00000100:00100000:1.0:1789760797.277645:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727386569712 time 727386309296 state 0x1 ref 1 00000100:00100000:1.0:1789760797.277688:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727386569712 time 727402294295 state 0x0 ref 1 00000100:00100000:1.0:1789760797.277702:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503945856:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.277844:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81780736, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27008 00010000:00010000:1.0:1789760797.277856:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.277878:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.277883:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.277891:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.277909:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.286505:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987273 is committed 00002000:00010000:1.0:1789760797.286856:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81780736 -> 81784832 00002000:00010000:1.0:1789760797.286864:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159752 -> 159760 00010000:00010000:1.0:1789760797.286878:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.286890:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.286899:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.286906:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.286910:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.286918:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.286938:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.286946:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f29fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81780736->81784831] (req 81780736->81784831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.287049:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503945856:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9344us (10184us total) trans 4294987273 rc 0/0 00000100:00100000:1.0:1789760797.287060:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19977 00000100:00100000:1.0:1789760797.287069:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727386569712 time 727402294295 state 0x2 ref 0 00000100:00100000:1.0:1789760797.306622:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.306777:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503945984 00000100:00100000:0.0:1789760797.306791:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503945984 00000100:00100000:0.0:1789760797.307010:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727386569712 time 727402294295 state 0x0 ref 1 00000100:00100000:0.0:1789760797.307030:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727402554711 time 727402294295 state 0x1 ref 1 00000100:00100000:0.0:1789760797.307073:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727402554711 time 727431677713 state 0x0 ref 1 00000100:00100000:0.0:1789760797.307090:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503945984:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.307160:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81784832, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27010 00010000:00010000:0.0:1789760797.307171:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.307194:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.307199:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.308015:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.308050:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.317081:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987274 is committed 00002000:00010000:0.0:1789760797.319304:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81784832 -> 81788928 00002000:00010000:0.0:1789760797.319319:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159760 -> 159768 00010000:00010000:0.0:1789760797.319338:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.319355:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.319366:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.319375:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.319382:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.319445:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.319484:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.320232:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81784832->81788927] (req 81784832->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.320381:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503945984:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13289us (13768us total) trans 4294987274 rc 0/0 00000100:00100000:0.0:1789760797.320462:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19978 00000100:00100000:0.0:1789760797.320475:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727402554711 time 727431677713 state 0x2 ref 0 00000100:00100000:2.0:1789760797.327988:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.330692:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503946112 00000100:00100000:1.0:1789760797.330707:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503946112 00000100:00100000:1.0:1789760797.330770:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727402554711 time 727431677713 state 0x0 ref 1 00000100:00100000:1.0:1789760797.330795:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727431938129 time 727431677713 state 0x1 ref 1 00000100:00100000:1.0:1789760797.330832:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727431938129 time 727455437887 state 0x0 ref 1 00000100:00100000:1.0:1789760797.330844:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946112:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.330895:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81788928, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27018 00010000:00010000:1.0:1789760797.330904:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.330921:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.330925:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.330932:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.330952:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.337380:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987275 is committed 00002000:00010000:2.0:1789760797.337662:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81788928 -> 81793024 00002000:00010000:2.0:1789760797.337668:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159768 -> 159776 00010000:00010000:2.0:1789760797.337686:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.337826:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.337837:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.337845:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.337850:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.337860:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.337884:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.337893:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006afffecf/0xfa2da65b083f2a09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81788928->81793023] (req 81788928->81793023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.338242:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946112:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7393us (10256us total) trans 4294987275 rc 0/0 00000100:00100000:2.0:1789760797.338257:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19979 00000100:00100000:2.0:1789760797.338271:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727431938129 time 727455437887 state 0x2 ref 0 00000100:00100000:2.0:1789760797.345233:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.345470:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503946240 00000100:00100000:0.0:1789760797.345486:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503946240 00000100:00100000:0.0:1789760797.345603:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727431938129 time 727455437887 state 0x0 ref 1 00000100:00100000:0.0:1789760797.345626:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727455698303 time 727455437887 state 0x1 ref 1 00000100:00100000:0.0:1789760797.345657:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727455698303 time 727470263519 state 0x0 ref 1 00000100:00100000:0.0:1789760797.345671:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503946240:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.345745:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81793024, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27020 00010000:00010000:0.0:1789760797.345758:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.345778:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.345784:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.345793:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.345811:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.353975:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987276 is committed 00002000:00010000:2.0:1789760797.354162:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81793024 -> 81797120 00002000:00010000:2.0:1789760797.354170:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159776 -> 159784 00010000:00010000:2.0:1789760797.354203:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.354218:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.354228:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.354235:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.354240:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.354252:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.354282:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.354290:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2a10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81793024->81797119] (req 81793024->81797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.354644:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503946240:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8967us (9420us total) trans 4294987276 rc 0/0 00000100:00100000:2.0:1789760797.354660:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19980 00000100:00100000:2.0:1789760797.354674:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727455698303 time 727470263519 state 0x2 ref 0 00000100:00100000:1.0:1789760797.362687:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.362833:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503946368 00000100:00100000:2.0:1789760797.362848:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503946368 00000100:00100000:2.0:1789760797.362923:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727455698303 time 727470263519 state 0x0 ref 1 00000100:00100000:2.0:1789760797.362943:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727470523935 time 727470263519 state 0x1 ref 1 00000100:00100000:2.0:1789760797.362982:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727470523935 time 727487588358 state 0x0 ref 1 00000100:00100000:2.0:1789760797.362996:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946368:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.363064:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81797120, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27028 00010000:00010000:2.0:1789760797.363077:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.363098:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.363104:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.363112:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.363130:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.368108:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987277 is committed 00002000:00010000:2.0:1789760797.372654:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81797120 -> 81801216 00002000:00010000:2.0:1789760797.372668:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159784 -> 159792 00010000:00010000:2.0:1789760797.372685:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.372701:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.372710:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.372718:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.372724:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.372735:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.372761:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.372768:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2a17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81797120->81801215] (req 81797120->81801215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.372956:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946368:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9955us (10278us total) trans 4294987277 rc 0/0 00000100:00100000:2.0:1789760797.372970:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19981 00000100:00100000:2.0:1789760797.372982:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727470523935 time 727487588358 state 0x2 ref 0 00000100:00100000:1.0:1789760797.377640:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.377746:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503946496 00000100:00100000:0.0:1789760797.377758:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503946496 00000100:00100000:0.0:1789760797.377826:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727470523935 time 727487588358 state 0x0 ref 1 00000100:00100000:0.0:1789760797.377844:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727487848774 time 727487588358 state 0x1 ref 1 00000100:00100000:0.0:1789760797.377905:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727487848774 time 727502511495 state 0x0 ref 1 00000100:00100000:0.0:1789760797.377920:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503946496:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.377978:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81801216, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27030 00010000:00010000:0.0:1789760797.377989:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.378008:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.378013:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.378022:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.378039:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.383641:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987278 is committed 00002000:00010000:0.0:1789760797.384118:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81801216 -> 81805312 00002000:00010000:0.0:1789760797.384126:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159792 -> 159800 00010000:00010000:0.0:1789760797.384143:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.384160:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.384170:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.384258:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.384264:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.384274:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.384299:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.384307:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81801216->81805311] (req 81801216->81805311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.388364:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503946496:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10423us (10714us total) trans 4294987278 rc 0/0 00000100:00100000:1.0:1789760797.388384:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19982 00000100:00100000:1.0:1789760797.391691:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727487848774 time 727502511495 state 0x2 ref 0 00000100:00100000:0.0:1789760797.392446:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.393376:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503946624 00000100:00100000:1.0:1789760797.393445:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503946624 00000100:00100000:1.0:1789760797.393658:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727487848774 time 727502511495 state 0x0 ref 1 00000100:00100000:1.0:1789760797.393679:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727502771911 time 727502511495 state 0x1 ref 1 00000100:00100000:1.0:1789760797.393716:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727502771911 time 727518322239 state 0x0 ref 1 00000100:00100000:1.0:1789760797.393732:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946624:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.393807:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81805312, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27038 00010000:00010000:1.0:1789760797.393818:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.393838:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.393844:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.393852:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.393870:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.394545:0:3638:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760797.394570:0:3638:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b5dff566300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523617280:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760797.394655:0:3638:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.394740:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699523617280 00000100:00100000:0.0:1789760797.394746:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699523617280 00000100:00100000:0.0:1789760797.394791:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:0.0:1789760797.394800:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff564380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523617280:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760797.394841:0:6518:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:0.0:1789760797.394903:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff564380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3638:x1876699523617280:12345-0@lo:41:osp-pre-0-1.0 Request processed in 103us (247us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.394911:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:0.0:1789760797.394969:0:3638:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b5dff566300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3638:1876699523617280:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789760797.403290:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987279 is committed 00002000:00010000:2.0:1789760797.403714:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81805312 -> 81809408 00002000:00010000:2.0:1789760797.403721:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159800 -> 159808 00010000:00010000:2.0:1789760797.403739:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.403756:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.403766:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.403775:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.403781:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.403792:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.403817:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.403825:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2a25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81805312->81809407] (req 81805312->81809407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.403998:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946624:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10260us (11548us total) trans 4294987279 rc 0/0 00000100:00100000:2.0:1789760797.404012:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19983 00000100:00100000:2.0:1789760797.404028:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727502771911 time 727518322239 state 0x2 ref 0 00000100:00100000:2.0:1789760797.409657:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.409775:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503946752 00000100:00100000:0.0:1789760797.409789:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503946752 00000100:00100000:0.0:1789760797.409872:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727502771911 time 727518322239 state 0x0 ref 1 00000100:00100000:0.0:1789760797.409892:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727518582655 time 727518322239 state 0x1 ref 1 00000100:00100000:0.0:1789760797.409932:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727518582655 time 727534537698 state 0x0 ref 1 00000100:00100000:0.0:1789760797.409946:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503946752:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.410011:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81809408, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27040 00010000:00010000:0.0:1789760797.410022:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.410042:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.410047:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.410054:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.410070:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.416620:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987280 is committed 00002000:00010000:2.0:1789760797.416921:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81809408 -> 81813504 00002000:00010000:2.0:1789760797.416928:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159808 -> 159816 00010000:00010000:2.0:1789760797.416946:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.416961:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.416971:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.416978:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.416984:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.416994:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417017:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417025:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2a2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81809408->81813503] (req 81809408->81813503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.417360:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503946752:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7408us (7714us total) trans 4294987280 rc 0/0 00000100:00100000:2.0:1789760797.417374:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19984 00000100:00100000:2.0:1789760797.417387:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727518582655 time 727534537698 state 0x2 ref 0 00000100:00100000:0.0:1789760797.425936:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.426193:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503946880 00000100:00100000:2.0:1789760797.426207:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503946880 00000100:00100000:2.0:1789760797.426569:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727518582655 time 727534537698 state 0x0 ref 1 00000100:00100000:2.0:1789760797.426589:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727534798114 time 727534537698 state 0x1 ref 1 00000100:00100000:2.0:1789760797.426620:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727534798114 time 727551227175 state 0x0 ref 1 00000100:00100000:2.0:1789760797.426635:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff566300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946880:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.426707:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81813504, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27048 00010000:00010000:2.0:1789760797.426719:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.426739:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.426745:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.426752:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.429137:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.434446:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987281 is committed 00002000:00010000:2.0:1789760797.434672:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81813504 -> 81817600 00002000:00010000:2.0:1789760797.434678:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159816 -> 159824 00010000:00010000:2.0:1789760797.434696:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.434713:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.434723:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.434732:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.434738:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.434750:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.434777:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.434785:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2a33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81813504->81817599] (req 81813504->81817599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.434912:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff566300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503946880:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8270us (8985us total) trans 4294987281 rc 0/0 00000100:00100000:2.0:1789760797.434925:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19985 00000100:00100000:2.0:1789760797.434938:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727534798114 time 727551227175 state 0x2 ref 0 00000100:00100000:0.0:1789760797.439931:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.440042:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947008 00000100:00100000:2.0:1789760797.440055:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947008 00000100:00100000:2.0:1789760797.440136:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727534798114 time 727551227175 state 0x0 ref 1 00000100:00100000:2.0:1789760797.440156:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727551487591 time 727551227175 state 0x1 ref 1 00000100:00100000:2.0:1789760797.440197:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727551487591 time 727564803520 state 0x0 ref 1 00000100:00100000:2.0:1789760797.440210:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947008:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.440279:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81817600, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27050 00010000:00010000:2.0:1789760797.440291:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.440311:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.440316:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.440324:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.440341:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.445441:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987282 is committed 00002000:00010000:1.0:1789760797.446807:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81817600 -> 81821696 00002000:00010000:1.0:1789760797.446815:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159824 -> 159832 00010000:00010000:1.0:1789760797.446831:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.446846:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.446855:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.446863:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.446868:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.446879:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.446902:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.446915:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2a3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81817600->81821695] (req 81817600->81821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.447444:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947008:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7173us (7467us total) trans 4294987282 rc 0/0 00000100:00100000:1.0:1789760797.447468:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19986 00000100:00100000:1.0:1789760797.447482:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727551487591 time 727564803520 state 0x2 ref 0 00000100:00100000:1.0:1789760797.452944:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.453309:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947136 00000100:00100000:0.0:1789760797.453324:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947136 00000100:00100000:0.0:1789760797.453424:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727551487591 time 727564803520 state 0x0 ref 1 00000100:00100000:0.0:1789760797.453442:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727565063936 time 727564803520 state 0x1 ref 1 00000100:00100000:0.0:1789760797.453490:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727565063936 time 727578095786 state 0x0 ref 1 00000100:00100000:0.0:1789760797.453505:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7df3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503947136:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.453564:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81821696, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27058 00010000:00010000:0.0:1789760797.453575:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.453594:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.453599:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.453605:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.453621:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.459236:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987283 is committed 00002000:00010000:0.0:1789760797.459600:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81821696 -> 81825792 00002000:00010000:0.0:1789760797.459607:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159832 -> 159840 00010000:00010000:0.0:1789760797.459625:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.459640:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.459655:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.459663:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.459669:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.459680:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.459707:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.459715:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81821696->81825791] (req 81821696->81825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.459844:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7df3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503947136:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6333us (6908us total) trans 4294987283 rc 0/0 00000100:00100000:0.0:1789760797.459856:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19987 00000100:00100000:0.0:1789760797.459869:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727565063936 time 727578095786 state 0x2 ref 0 00000100:00100000:0.0:1789760797.463240:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.463347:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947264 00000100:00100000:2.0:1789760797.463360:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947264 00000100:00100000:2.0:1789760797.463746:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727565063936 time 727578095786 state 0x0 ref 1 00000100:00100000:2.0:1789760797.463765:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727578356202 time 727578095786 state 0x1 ref 1 00000100:00100000:2.0:1789760797.463792:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727578356202 time 727588399162 state 0x0 ref 1 00000100:00100000:2.0:1789760797.463807:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947264:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.463867:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81825792, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27060 00010000:00010000:2.0:1789760797.463877:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.463896:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.463901:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.463910:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.463926:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.469336:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987284 is committed 00002000:00010000:2.0:1789760797.469709:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81825792 -> 81829888 00002000:00010000:2.0:1789760797.469718:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159840 -> 159848 00010000:00010000:2.0:1789760797.469737:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.469752:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.469772:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.469784:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.469791:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.469800:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.469826:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.469835:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81825792->81829887] (req 81825792->81829887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.469968:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947264:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6158us (6736us total) trans 4294987284 rc 0/0 00000100:00100000:2.0:1789760797.469981:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19988 00000100:00100000:2.0:1789760797.469993:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727578356202 time 727588399162 state 0x2 ref 0 00000100:00100000:2.0:1789760797.474423:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.474535:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947392 00000100:00100000:1.0:1789760797.474548:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947392 00000100:00100000:1.0:1789760797.474628:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727578356202 time 727588399162 state 0x0 ref 1 00000100:00100000:1.0:1789760797.474647:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727588659578 time 727588399162 state 0x1 ref 1 00000100:00100000:1.0:1789760797.474674:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727588659578 time 727599280384 state 0x0 ref 1 00000100:00100000:1.0:1789760797.474700:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503947392:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.474772:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81829888, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27068 00010000:00010000:1.0:1789760797.474784:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.474804:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.474809:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.474817:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.474836:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.481462:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987285 is committed 00002000:00010000:1.0:1789760797.481882:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81829888 -> 81833984 00002000:00010000:1.0:1789760797.481889:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159848 -> 159856 00010000:00010000:1.0:1789760797.481907:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.481923:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.481932:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.481940:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.481945:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.481956:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.481983:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.481991:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002503319/0xfa2da65b083f2a4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81829888->81833983] (req 81829888->81833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.482150:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503947392:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7447us (7767us total) trans 4294987285 rc 0/0 00000100:00100000:1.0:1789760797.482164:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19989 00000100:00100000:1.0:1789760797.482190:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727588659578 time 727599280384 state 0x2 ref 0 00000100:00100000:1.0:1789760797.486254:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.486357:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947520 00000100:00100000:2.0:1789760797.486369:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947520 00000100:00100000:2.0:1789760797.486484:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727588659578 time 727599280384 state 0x0 ref 1 00000100:00100000:2.0:1789760797.486513:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727599540800 time 727599280384 state 0x1 ref 1 00000100:00100000:2.0:1789760797.486549:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727599540800 time 727611154812 state 0x0 ref 1 00000100:00100000:2.0:1789760797.486563:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503947520:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.486638:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81833984, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27070 00010000:00010000:2.0:1789760797.486649:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.486669:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.486676:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.486859:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.486879:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.492161:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987286 is committed 00002000:00010000:2.0:1789760797.492360:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81833984 -> 81838080 00002000:00010000:2.0:1789760797.492365:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159856 -> 159864 00010000:00010000:2.0:1789760797.492380:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.492420:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.492430:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.492439:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.492444:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.492465:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.492501:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.492509:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81833984->81838079] (req 81833984->81838079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.493089:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503947520:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6520us (6840us total) trans 4294987286 rc 0/0 00000100:00100000:2.0:1789760797.493104:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19990 00000100:00100000:2.0:1789760797.493116:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727599540800 time 727611154812 state 0x2 ref 0 00000100:00100000:1.0:1789760797.509644:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.511138:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947648 00000100:00100000:0.0:1789760797.511157:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947648 00000100:00100000:0.0:1789760797.511312:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727599540800 time 727611154812 state 0x0 ref 1 00000100:00100000:0.0:1789760797.511332:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727611415228 time 727611154812 state 0x1 ref 1 00000100:00100000:0.0:1789760797.511359:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727611415228 time 727635965935 state 0x0 ref 1 00000100:00100000:0.0:1789760797.511373:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503947648:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.511484:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81838080, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27078 00010000:00010000:0.0:1789760797.511495:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.511812:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.511819:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.511828:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.511847:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.516871:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987287 is committed 00002000:00010000:0.0:1789760797.517005:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81838080 -> 81842176 00002000:00010000:0.0:1789760797.517012:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159864 -> 159872 00010000:00010000:0.0:1789760797.517028:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.517041:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.517056:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.517063:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.517068:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.517078:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.517101:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.517109:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2a5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81838080->81842175] (req 81838080->81842175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.517229:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503947648:12345-192.168.203.58@tcp:4:dd.0 Request processed in 5853us (7591us total) trans 4294987287 rc 0/0 00000100:00100000:0.0:1789760797.517241:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19991 00000100:00100000:0.0:1789760797.517253:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727611415228 time 727635965935 state 0x2 ref 0 00000100:00100000:0.0:1789760797.524564:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.524660:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947776 00000100:00100000:2.0:1789760797.524674:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947776 00000100:00100000:2.0:1789760797.524762:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727611415228 time 727635965935 state 0x0 ref 1 00000100:00100000:2.0:1789760797.524781:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727636226351 time 727635965935 state 0x1 ref 1 00000100:00100000:2.0:1789760797.524821:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727636226351 time 727649426882 state 0x0 ref 1 00000100:00100000:2.0:1789760797.524835:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947776:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.524903:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81842176, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27080 00010000:00010000:2.0:1789760797.524914:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.524933:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.524938:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.524945:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.524963:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.531019:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987288 is committed 00002000:00010000:2.0:1789760797.531584:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81842176 -> 81846272 00002000:00010000:2.0:1789760797.531591:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159872 -> 159880 00010000:00010000:2.0:1789760797.531611:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.531626:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.531637:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.531646:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.531652:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.531663:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.531758:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.531768:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81842176->81846271] (req 81842176->81846271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.532333:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947776:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7491us (7775us total) trans 4294987288 rc 0/0 00000100:00100000:2.0:1789760797.532349:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19992 00000100:00100000:2.0:1789760797.532362:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727636226351 time 727649426882 state 0x2 ref 0 00000100:00100000:1.0:1789760797.536630:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.537635:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503947904 00000100:00100000:2.0:1789760797.537652:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503947904 00000100:00100000:2.0:1789760797.537733:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727636226351 time 727649426882 state 0x0 ref 1 00000100:00100000:2.0:1789760797.537753:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727649687298 time 727649426882 state 0x1 ref 1 00000100:00100000:2.0:1789760797.537788:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727649687298 time 727662394009 state 0x0 ref 1 00000100:00100000:2.0:1789760797.537803:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc87c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947904:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.537872:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81846272, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27088 00010000:00010000:2.0:1789760797.537883:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.537903:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.537909:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.537917:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.537934:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.544039:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987289 is committed 00002000:00010000:2.0:1789760797.544201:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81846272 -> 81850368 00002000:00010000:2.0:1789760797.544209:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159880 -> 159888 00010000:00010000:2.0:1789760797.544225:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544240:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544249:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544263:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.544268:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544278:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544301:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544309:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81846272->81850367] (req 81846272->81850367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.544490:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc87c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503947904:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6683us (7868us total) trans 4294987289 rc 0/0 00000100:00100000:2.0:1789760797.544501:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19993 00000100:00100000:2.0:1789760797.544513:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727649687298 time 727662394009 state 0x2 ref 0 00000100:00100000:1.0:1789760797.549951:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.550406:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948032 00000100:00100000:2.0:1789760797.550421:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948032 00000100:00100000:2.0:1789760797.550514:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727649687298 time 727662394009 state 0x0 ref 1 00000100:00100000:2.0:1789760797.550534:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727662654425 time 727662394009 state 0x1 ref 1 00000100:00100000:2.0:1789760797.550569:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727662654425 time 727675175007 state 0x0 ref 1 00000100:00100000:2.0:1789760797.550587:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948032:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.550656:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81850368, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27090 00010000:00010000:2.0:1789760797.550668:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.550689:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.550694:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.550703:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.550720:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.566122:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987290 is committed 00002000:00010000:2.0:1789760797.566366:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81850368 -> 81854464 00002000:00010000:2.0:1789760797.566372:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159888 -> 159896 00010000:00010000:2.0:1789760797.566431:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.566911:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.566925:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.566934:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.566940:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.566952:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.566978:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.566986:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81850368->81854463] (req 81850368->81854463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.567122:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948032:12345-192.168.203.58@tcp:4:dd.0 Request processed in 16531us (17179us total) trans 4294987290 rc 0/0 00000100:00100000:2.0:1789760797.567134:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19994 00000100:00100000:2.0:1789760797.567147:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727662654425 time 727675175007 state 0x2 ref 0 00000100:00100000:0.0:1789760797.573709:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.574317:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948160 00000100:00100000:2.0:1789760797.574333:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948160 00000100:00100000:2.0:1789760797.574598:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727662654425 time 727675175007 state 0x0 ref 1 00000100:00100000:2.0:1789760797.574619:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727675435423 time 727675175007 state 0x1 ref 1 00000100:00100000:2.0:1789760797.574661:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727675435423 time 727699267369 state 0x0 ref 1 00000100:00100000:2.0:1789760797.574676:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948160:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.574741:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81854464, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27098 00010000:00010000:2.0:1789760797.574753:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.574772:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.574778:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.574785:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.574800:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.583853:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987291 is committed 00002000:00010000:2.0:1789760797.584018:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81854464 -> 81858560 00002000:00010000:2.0:1789760797.584025:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159896 -> 159904 00010000:00010000:2.0:1789760797.584043:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.584059:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.584069:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.584078:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.584083:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.584094:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.584121:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.584130:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2a79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81854464->81858559] (req 81854464->81858559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.584266:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948160:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9585us (10566us total) trans 4294987291 rc 0/0 00000100:00100000:2.0:1789760797.584278:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19995 00000100:00100000:2.0:1789760797.584292:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727675435423 time 727699267369 state 0x2 ref 0 00000100:00100000:1.0:1789760797.589688:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.591167:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948288 00000100:00100000:0.0:1789760797.591187:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948288 00000100:00100000:0.0:1789760797.591332:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727675435423 time 727699267369 state 0x0 ref 1 00000100:00100000:0.0:1789760797.591351:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727699527785 time 727699267369 state 0x1 ref 1 00000100:00100000:0.0:1789760797.591404:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727699527785 time 727715996343 state 0x0 ref 1 00000100:00100000:0.0:1789760797.591420:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948288:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.591498:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81858560, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270a0 00010000:00010000:0.0:1789760797.591509:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.591530:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.591535:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.591543:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.591560:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.601046:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987292 is committed 00002000:00010000:0.0:1789760797.601209:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81858560 -> 81862656 00002000:00010000:0.0:1789760797.601215:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159904 -> 159912 00010000:00010000:0.0:1789760797.601232:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.601254:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.601265:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.601273:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.601279:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.601289:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.601313:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.601322:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81858560->81862655] (req 81858560->81862655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.601537:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948288:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10112us (11856us total) trans 4294987292 rc 0/0 00000100:00100000:0.0:1789760797.601550:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19996 00000100:00100000:0.0:1789760797.601563:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727699527785 time 727715996343 state 0x2 ref 0 00000100:00100000:1.0:1789760797.605259:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.605540:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948416 00000100:00100000:2.0:1789760797.605554:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948416 00000100:00100000:2.0:1789760797.605632:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727699527785 time 727715996343 state 0x0 ref 1 00000100:00100000:2.0:1789760797.605653:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727716256759 time 727715996343 state 0x1 ref 1 00000100:00100000:2.0:1789760797.605689:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727716256759 time 727730295116 state 0x0 ref 1 00000100:00100000:2.0:1789760797.605705:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503948416:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.605780:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81862656, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270a8 00010000:00010000:2.0:1789760797.605793:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.605819:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.605824:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.605832:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.605849:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.614076:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987293 is committed 00002000:00010000:3.0:1789760797.614364:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81862656 -> 81866752 00002000:00010000:3.0:1789760797.614370:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159912 -> 159920 00010000:00010000:3.0:1789760797.614412:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.614426:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.614436:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.614443:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760797.614458:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.614474:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.614796:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.614809:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2a87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81862656->81866751] (req 81862656->81866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.615008:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503948416:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9298us (9758us total) trans 4294987293 rc 0/0 00000100:00100000:3.0:1789760797.615022:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19997 00000100:00100000:3.0:1789760797.615034:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727716256759 time 727730295116 state 0x2 ref 0 00000100:00100000:2.0:1789760797.620084:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760797.620144:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948544 00000100:00100000:3.0:1789760797.620156:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948544 00000100:00100000:3.0:1789760797.620231:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727716256759 time 727730295116 state 0x0 ref 1 00000100:00100000:3.0:1789760797.620250:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727730555532 time 727730295116 state 0x1 ref 1 00000100:00100000:3.0:1789760797.620286:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727730555532 time 727744892788 state 0x0 ref 1 00000100:00100000:3.0:1789760797.620302:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503948544:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760797.620371:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81866752, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270b0 00010000:00010000:3.0:1789760797.620383:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.620440:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760797.620611:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.620623:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.620643:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.629193:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987294 is committed 00002000:00010000:2.0:1789760797.631266:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81866752 -> 81870848 00002000:00010000:2.0:1789760797.631279:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159920 -> 159928 00010000:00010000:2.0:1789760797.631298:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.631317:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.631326:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.631334:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.631340:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.631350:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.631376:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.631385:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079839492/0xfa2da65b083f2a8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81866752->81870847] (req 81866752->81870847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.631577:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cd24a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503948544:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11270us (11495us total) trans 4294987294 rc 0/0 00000100:00100000:2.0:1789760797.631590:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19998 00000100:00100000:2.0:1789760797.631602:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727730555532 time 727744892788 state 0x2 ref 0 00000100:00100000:1.0:1789760797.645786:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.648813:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948672 00000100:00100000:0.0:1789760797.648837:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948672 00000100:00100000:0.0:1789760797.649267:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727730555532 time 727744892788 state 0x0 ref 1 00000100:00100000:0.0:1789760797.649287:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727745153204 time 727744892788 state 0x1 ref 1 00000100:00100000:0.0:1789760797.651471:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727745153204 time 727776047486 state 0x0 ref 1 00000100:00100000:0.0:1789760797.651535:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948672:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.651616:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81870848, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270b8 00010000:00010000:0.0:1789760797.651628:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.651648:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.651653:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.651660:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.651682:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.660632:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987295 is committed 00002000:00010000:0.0:1789760797.660795:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81870848 -> 81874944 00002000:00010000:0.0:1789760797.660801:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159928 -> 159936 00010000:00010000:0.0:1789760797.660819:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.660833:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.660847:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.660855:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.660862:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.660873:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.660899:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.660908:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2a95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81870848->81874943] (req 81870848->81874943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.661192:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948672:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9675us (15414us total) trans 4294987295 rc 0/0 00000100:00100000:0.0:1789760797.661206:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 19999 00000100:00100000:0.0:1789760797.661219:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727745153204 time 727776047486 state 0x2 ref 0 00000100:00100000:1.0:1789760797.664644:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.664783:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948800 00000100:00100000:0.0:1789760797.664797:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948800 00000100:00100000:0.0:1789760797.664880:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727745153204 time 727776047486 state 0x0 ref 1 00000100:00100000:0.0:1789760797.664902:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727776307902 time 727776047486 state 0x1 ref 1 00000100:00100000:0.0:1789760797.664938:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727776307902 time 727789544871 state 0x0 ref 1 00000100:00100000:0.0:1789760797.664952:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948800:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.665027:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81874944, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270c0 00010000:00010000:0.0:1789760797.665038:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.665057:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.665063:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.665070:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.665086:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.672728:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987296 is committed 00002000:00010000:0.0:1789760797.676730:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81874944 -> 81879040 00002000:00010000:0.0:1789760797.676746:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159936 -> 159944 00010000:00010000:0.0:1789760797.676765:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.676781:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.676791:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.676807:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.676812:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.676822:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.676848:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.676857:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2a9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81874944->81879039] (req 81874944->81879039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.676998:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc743fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503948800:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12040us (12359us total) trans 4294987296 rc 0/0 00000100:00100000:0.0:1789760797.677010:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20000 00000100:00100000:0.0:1789760797.677022:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727776307902 time 727789544871 state 0x2 ref 0 00000100:00100000:2.0:1789760797.682518:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760797.682658:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503948928 00000100:00100000:3.0:1789760797.682673:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503948928 00000100:00100000:3.0:1789760797.682746:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727776307902 time 727789544871 state 0x0 ref 1 00000100:00100000:3.0:1789760797.682764:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727789805287 time 727789544871 state 0x1 ref 1 00000100:00100000:3.0:1789760797.682800:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727789805287 time 727807406457 state 0x0 ref 1 00000100:00100000:3.0:1789760797.682814:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503948928:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760797.682881:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81879040, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270c8 00010000:00010000:3.0:1789760797.682893:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.682924:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760797.682937:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.682945:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.682970:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.689913:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987297 is committed 00002000:00010000:3.0:1789760797.690157:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81879040 -> 81883136 00002000:00010000:3.0:1789760797.690164:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159944 -> 159952 00010000:00010000:3.0:1789760797.690231:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.690248:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.690257:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.690265:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760797.690271:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.690280:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.690304:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.690318:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2aa3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81879040->81883135] (req 81879040->81883135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.690526:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503948928:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7707us (8130us total) trans 4294987297 rc 0/0 00000100:00100000:3.0:1789760797.690539:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20001 00000100:00100000:3.0:1789760797.690551:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727789805287 time 727807406457 state 0x2 ref 0 00000100:00100000:1.0:1789760797.699784:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.699940:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949056 00000100:00100000:2.0:1789760797.699955:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949056 00000100:00100000:2.0:1789760797.700216:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727789805287 time 727807406457 state 0x0 ref 1 00000100:00100000:2.0:1789760797.700236:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727807666873 time 727807406457 state 0x1 ref 1 00000100:00100000:2.0:1789760797.700285:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727807666873 time 727824891806 state 0x0 ref 1 00000100:00100000:2.0:1789760797.700300:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949056:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.700369:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81883136, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270d0 00010000:00010000:2.0:1789760797.700381:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.700621:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.700626:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.700635:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.700652:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.707215:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987298 is committed 00002000:00010000:2.0:1789760797.707466:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81883136 -> 81887232 00002000:00010000:2.0:1789760797.707472:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159952 -> 159960 00010000:00010000:2.0:1789760797.707489:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.707504:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.707514:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.707522:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.707528:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.707538:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.707568:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.707576:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089d74ba8/0xfa2da65b083f2aaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81883136->81887231] (req 81883136->81887231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.707745:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949056:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7439us (7966us total) trans 4294987298 rc 0/0 00000100:00100000:2.0:1789760797.707756:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20002 00000100:00100000:2.0:1789760797.707768:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727807666873 time 727824891806 state 0x2 ref 0 00000100:00100000:1.0:1789760797.714229:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.714952:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949184 00000100:00100000:0.0:1789760797.714969:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949184 00000100:00100000:0.0:1789760797.715158:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727807666873 time 727824891806 state 0x0 ref 1 00000100:00100000:0.0:1789760797.715178:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727825152222 time 727824891806 state 0x1 ref 1 00000100:00100000:0.0:1789760797.715213:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727825152222 time 727839819393 state 0x0 ref 1 00000100:00100000:0.0:1789760797.715228:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949184:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.715301:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81887232, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270d8 00010000:00010000:0.0:1789760797.715313:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.715333:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.715339:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.715348:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.715440:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.719166:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987299 is committed 00002000:00010000:1.0:1789760797.719849:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81887232 -> 81891328 00002000:00010000:1.0:1789760797.719857:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159960 -> 159968 00010000:00010000:1.0:1789760797.719874:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.719890:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.719899:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.719909:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.719915:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.719925:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.719948:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.719955:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e6376d9/0xfa2da65b083f2ab1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81887232->81891327] (req 81887232->81891327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.720155:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949184:12345-192.168.203.58@tcp:4:dd.0 Request processed in 4871us (5882us total) trans 4294987299 rc 0/0 00000100:00100000:1.0:1789760797.720170:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20003 00000100:00100000:1.0:1789760797.720239:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727825152222 time 727839819393 state 0x2 ref 0 00000100:00100000:2.0:1789760797.724067:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.724162:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949312 00000100:00100000:0.0:1789760797.724175:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949312 00000100:00100000:0.0:1789760797.724254:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727825152222 time 727839819393 state 0x0 ref 1 00000100:00100000:0.0:1789760797.724274:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727840079809 time 727839819393 state 0x1 ref 1 00000100:00100000:0.0:1789760797.724310:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727840079809 time 727848915509 state 0x0 ref 1 00000100:00100000:0.0:1789760797.724324:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949312:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.724519:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81891328, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270e0 00010000:00010000:0.0:1789760797.724532:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.724552:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760797.724557:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.724565:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.724583:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.730694:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987300 is committed 00002000:00010000:1.0:1789760797.730889:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81891328 -> 81895424 00002000:00010000:1.0:1789760797.730990:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159968 -> 159976 00010000:00010000:1.0:1789760797.731010:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.731026:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.731041:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.731050:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3db600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.731056:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.731066:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.731089:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.731097:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a9b5f84/0xfa2da65b083f2ab8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81891328->81895423] (req 81891328->81895423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.731256:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949312:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6928us (7198us total) trans 4294987300 rc 0/0 00000100:00100000:1.0:1789760797.731266:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20004 00000100:00100000:1.0:1789760797.731278:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727840079809 time 727848915509 state 0x2 ref 0 00000100:00100000:2.0:1789760797.734989:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760797.735285:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949440 00000100:00100000:3.0:1789760797.735298:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949440 00000100:00100000:3.0:1789760797.735370:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727840079809 time 727848915509 state 0x0 ref 1 00000100:00100000:3.0:1789760797.735470:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727849175925 time 727848915509 state 0x1 ref 1 00000100:00100000:3.0:1789760797.735540:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727849175925 time 727860146193 state 0x0 ref 1 00000100:00100000:3.0:1789760797.735556:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503949440:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.735640:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81895424, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270e8 00010000:00010000:1.0:1789760797.735653:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.735680:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.735685:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.735692:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.735707:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.744500:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987301 is committed 00002000:00010000:1.0:1789760797.745459:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81895424 -> 81899520 00002000:00010000:1.0:1789760797.745469:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159976 -> 159984 00010000:00010000:1.0:1789760797.745488:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.745502:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.745512:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.745521:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.745527:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.745538:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.745566:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.745574:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2abf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81895424->81899519] (req 81895424->81899519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.745702:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503949440:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10142us (10721us total) trans 4294987301 rc 0/0 00000100:00100000:1.0:1789760797.745715:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20005 00000100:00100000:1.0:1789760797.745729:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727849175925 time 727860146193 state 0x2 ref 0 00000100:00100000:2.0:1789760797.753384:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760797.753618:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949568 00000100:00100000:3.0:1789760797.753633:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949568 00000100:00100000:3.0:1789760797.753948:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727849175925 time 727860146193 state 0x0 ref 1 00000100:00100000:3.0:1789760797.753982:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727860406609 time 727860146193 state 0x1 ref 1 00000100:00100000:3.0:1789760797.754030:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727860406609 time 727878636104 state 0x0 ref 1 00000100:00100000:3.0:1789760797.754057:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949568:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760797.754141:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81899520, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270f0 00010000:00010000:3.0:1789760797.754158:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.754443:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760797.754462:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.754474:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.754626:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.763691:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987302 is committed 00002000:00010000:3.0:1789760797.763902:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81899520 -> 81903616 00002000:00010000:3.0:1789760797.763909:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159984 -> 159992 00010000:00010000:3.0:1789760797.763928:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.763944:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.763954:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.763962:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760797.763968:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.763978:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.764003:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.764011:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ac6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81899520->81903615] (req 81899520->81903615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.764158:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949568:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10096us (10781us total) trans 4294987302 rc 0/0 00000100:00100000:3.0:1789760797.764170:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20006 00000100:00100000:3.0:1789760797.764184:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727860406609 time 727878636104 state 0x2 ref 0 00000100:00100000:2.0:1789760797.776539:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.776815:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949696 00000100:00100000:1.0:1789760797.776829:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949696 00000100:00100000:1.0:1789760797.776898:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727860406609 time 727878636104 state 0x0 ref 1 00000100:00100000:1.0:1789760797.776916:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727878896520 time 727878636104 state 0x1 ref 1 00000100:00100000:1.0:1789760797.776953:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727878896520 time 727901559661 state 0x0 ref 1 00000100:00100000:1.0:1789760797.776967:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949696:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.777143:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81903616, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x270f8 00010000:00010000:1.0:1789760797.777153:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.777229:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.777236:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.777244:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.777261:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.787481:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987303 is committed 00002000:00010000:1.0:1789760797.788847:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81903616 -> 81907712 00002000:00010000:1.0:1789760797.788860:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 159992 -> 160000 00010000:00010000:1.0:1789760797.788878:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.788894:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.789075:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.789087:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.789093:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.789104:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.789130:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.789145:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2acd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81903616->81907711] (req 81903616->81907711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.789566:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949696:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12585us (13025us total) trans 4294987303 rc 0/0 00000100:00100000:1.0:1789760797.789580:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20007 00000100:00100000:1.0:1789760797.789592:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727878896520 time 727901559661 state 0x2 ref 0 00000100:00100000:2.0:1789760797.799098:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.799222:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949824 00000100:00100000:1.0:1789760797.799235:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949824 00000100:00100000:1.0:1789760797.799323:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727878896520 time 727901559661 state 0x0 ref 1 00000100:00100000:1.0:1789760797.799342:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727901820077 time 727901559661 state 0x1 ref 1 00000100:00100000:1.0:1789760797.799381:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727901820077 time 727923986689 state 0x0 ref 1 00000100:00100000:1.0:1789760797.799441:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949824:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.799519:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81907712, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x27100 00010000:00010000:1.0:1789760797.799530:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.799550:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760797.799556:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.799564:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.799586:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.808861:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987304 is committed 00002000:00010000:0.0:1789760797.809053:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81907712 -> 81911808 00002000:00010000:0.0:1789760797.809060:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:0.0:1789760797.809063:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00002000:00010000:0.0:1789760797.809066:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160000 -> 160008 00010000:00010000:0.0:1789760797.809086:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.809102:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.809114:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.809123:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.809129:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.809141:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.809169:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.809247:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2ad4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81907712->81911807] (req 81907712->81911807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.809526:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949824:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10081us (10437us total) trans 4294987304 rc 0/0 00000100:00100000:0.0:1789760797.809540:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20008 00000100:00100000:0.0:1789760797.809553:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727901820077 time 727923986689 state 0x2 ref 0 00000100:00100000:3.0:1789760797.813238:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.814480:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503949952 00000100:00100000:2.0:1789760797.814566:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503949952 00000100:00100000:2.0:1789760797.814654:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727901820077 time 727923986689 state 0x0 ref 1 00000100:00100000:2.0:1789760797.814676:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727924247105 time 727923986689 state 0x1 ref 1 00000100:00100000:2.0:1789760797.814889:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727924247105 time 727939493586 state 0x0 ref 1 00000100:00100000:2.0:1789760797.814906:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949952:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.814978:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81911808, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27108 00010000:00010000:2.0:1789760797.814990:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.815012:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.815017:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.815026:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.815045:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.822789:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987305 is committed 00002000:00010000:2.0:1789760797.822957:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81911808 -> 81915904 00002000:00010000:2.0:1789760797.822964:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160008 -> 160016 00010000:00010000:2.0:1789760797.822982:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.822998:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.823010:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.823025:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.823031:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.823041:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.823067:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.823076:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2adb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81911808->81915903] (req 81911808->81915903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.823215:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503949952:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8305us (9985us total) trans 4294987305 rc 0/0 00000100:00100000:2.0:1789760797.823228:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20009 00000100:00100000:2.0:1789760797.823241:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727924247105 time 727939493586 state 0x2 ref 0 00000100:00100000:2.0:1789760797.834006:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760797.840817:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950080 00000100:00100000:3.0:1789760797.840838:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950080 00000100:00100000:3.0:1789760797.840928:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727924247105 time 727939493586 state 0x0 ref 1 00000100:00100000:3.0:1789760797.840947:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727939754002 time 727939493586 state 0x1 ref 1 00000100:00100000:3.0:1789760797.840991:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727939754002 time 727965597934 state 0x0 ref 1 00000100:00100000:3.0:1789760797.841007:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503950080:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760797.841076:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81915904, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27110 00010000:00010000:3.0:1789760797.841088:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.841106:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760797.841111:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.841118:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.841136:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.850860:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987306 is committed 00002000:00010000:2.0:1789760797.862320:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81915904 -> 81920000 00002000:00010000:2.0:1789760797.862336:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160016 -> 160024 00010000:00010000:2.0:1789760797.862362:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.862383:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.862421:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.862429:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.862435:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.862453:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.862478:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.862682:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009161946d/0xfa2da65b083f2ae2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81915904->81919999] (req 81915904->81919999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.862831:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503950080:12345-192.168.203.58@tcp:4:dd.0 Request processed in 21821us (28832us total) trans 4294987306 rc 0/0 00000100:00100000:2.0:1789760797.862842:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20010 00000100:00100000:2.0:1789760797.862853:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727939754002 time 727965597934 state 0x2 ref 0 00000100:00100000:1.0:1789760797.868438:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.868713:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950208 00000100:00100000:2.0:1789760797.868727:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950208 00000100:00100000:2.0:1789760797.868806:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727939754002 time 727965597934 state 0x0 ref 1 00000100:00100000:2.0:1789760797.868826:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727965858350 time 727965597934 state 0x1 ref 1 00000100:00100000:2.0:1789760797.868861:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727965858350 time 727993467842 state 0x0 ref 1 00000100:00100000:2.0:1789760797.868875:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503950208:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.868939:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81920000, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27118 00010000:00010000:2.0:1789760797.868952:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.868971:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760797.868976:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.868983:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.868999:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.878237:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987307 is committed 00002000:00010000:1.0:1789760797.883322:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81920000 -> 81924096 00002000:00010000:1.0:1789760797.883339:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160024 -> 160032 00010000:00010000:1.0:1789760797.883358:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.883372:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.883382:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.883480:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.883487:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.883498:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.883527:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.883541:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc092d78/0xfa2da65b083f2ae9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81920000->81924095] (req 81920000->81924095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.883743:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503950208:12345-192.168.203.58@tcp:4:dd.0 Request processed in 14862us (15358us total) trans 4294987307 rc 0/0 00000100:00100000:1.0:1789760797.883756:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20011 00000100:00100000:1.0:1789760797.883770:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727965858350 time 727993467842 state 0x2 ref 0 00000100:00100000:3.0:1789760797.901527:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.901644:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950336 00000100:00100000:2.0:1789760797.901658:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950336 00000100:00100000:2.0:1789760797.901732:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727965858350 time 727993467842 state 0x0 ref 1 00000100:00100000:2.0:1789760797.901752:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727993728258 time 727993467842 state 0x1 ref 1 00000100:00100000:2.0:1789760797.901790:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727993728258 time 728026395734 state 0x0 ref 1 00000100:00100000:2.0:1789760797.901805:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503950336:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760797.901871:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81924096, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27120 00010000:00010000:2.0:1789760797.901883:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.901904:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760797.901909:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.901917:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.901935:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.912553:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987308 is committed 00002000:00010000:2.0:1789760797.912718:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81924096 -> 81928192 00002000:00010000:2.0:1789760797.912725:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160032 -> 160040 00010000:00010000:2.0:1789760797.912741:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.912757:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.912767:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.912775:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.912781:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.912791:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.912817:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.912826:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81924096->81928191] (req 81924096->81928191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.912975:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503950336:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11165us (11459us total) trans 4294987308 rc 0/0 00000100:00100000:2.0:1789760797.912988:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20012 00000100:00100000:2.0:1789760797.913002:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727993728258 time 728026395734 state 0x2 ref 0 00000100:00100000:1.0:1789760797.929073:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760797.929200:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950464 00000100:00100000:2.0:1789760797.929214:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950464 00000100:00100000:2.0:1789760797.929301:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 727993728258 time 728026395734 state 0x0 ref 1 00000100:00100000:2.0:1789760797.929320:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728026656150 time 728026395734 state 0x1 ref 1 00000100:00100000:2.0:1789760797.929427:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728026656150 time 728053994666 state 0x0 ref 1 00000100:00100000:2.0:1789760797.929443:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503950464:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.929598:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81928192, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27128 00010000:00010000:0.0:1789760797.929611:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.929632:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760797.929637:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.929645:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.929662:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.937809:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987309 is committed 00002000:00010000:0.0:1789760797.938051:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81928192 -> 81932288 00002000:00010000:0.0:1789760797.938058:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160040 -> 160048 00010000:00010000:0.0:1789760797.938075:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.938091:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.938101:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.938108:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760797.938114:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.938125:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.938150:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.938158:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046d78603/0xfa2da65b083f2af7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81928192->81932287] (req 81928192->81932287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.939609:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503950464:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10155us (10538us total) trans 4294987309 rc 0/0 00000100:00100000:0.0:1789760797.939626:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20013 00000100:00100000:0.0:1789760797.939640:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728026656150 time 728053994666 state 0x2 ref 0 00000100:00100000:1.0:1789760797.945122:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760797.945322:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950592 00000100:00100000:0.0:1789760797.945335:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950592 00000100:00100000:0.0:1789760797.945499:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728026656150 time 728053994666 state 0x0 ref 1 00000100:00100000:0.0:1789760797.945519:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728054255082 time 728053994666 state 0x1 ref 1 00000100:00100000:0.0:1789760797.945603:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728054255082 time 728070208026 state 0x0 ref 1 00000100:00100000:0.0:1789760797.945618:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503950592:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760797.945681:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81932288, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27130 00010000:00010000:0.0:1789760797.945692:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.945711:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760797.945716:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.945723:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.945739:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.953236:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987310 is committed 00002000:00010000:1.0:1789760797.954098:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81932288 -> 81936384 00002000:00010000:1.0:1789760797.954109:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160048 -> 160056 00010000:00010000:1.0:1789760797.954128:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.954144:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.954153:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.954161:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760797.954167:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.954378:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.954489:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.954499:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2afe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81932288->81936383] (req 81932288->81936383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.954677:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503950592:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9054us (9563us total) trans 4294987310 rc 0/0 00000100:00100000:1.0:1789760797.954691:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20014 00000100:00100000:1.0:1789760797.954705:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728054255082 time 728070208026 state 0x2 ref 0 00000100:00100000:3.0:1789760797.963511:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760797.964776:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950720 00000100:00100000:1.0:1789760797.964793:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950720 00000100:00100000:1.0:1789760797.964882:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728054255082 time 728070208026 state 0x0 ref 1 00000100:00100000:1.0:1789760797.964901:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728070468442 time 728070208026 state 0x1 ref 1 00000100:00100000:1.0:1789760797.965018:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728070468442 time 728089623632 state 0x0 ref 1 00000100:00100000:1.0:1789760797.965035:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503950720:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760797.965101:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81936384, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27138 00010000:00010000:1.0:1789760797.965111:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.965133:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760797.965138:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.965145:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.965163:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.975741:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987311 is committed 00002000:00010000:2.0:1789760797.976147:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81936384 -> 81940480 00002000:00010000:2.0:1789760797.976154:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160056 -> 160064 00010000:00010000:2.0:1789760797.976305:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.976322:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.976332:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.976341:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760797.976347:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.976358:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.976386:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.976419:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2b05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81936384->81940479] (req 81936384->81940479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.976654:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8946a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503950720:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11615us (14423us total) trans 4294987311 rc 0/0 00000100:00100000:2.0:1789760797.976666:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20015 00000100:00100000:2.0:1789760797.976679:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728070468442 time 728089623632 state 0x2 ref 0 00000100:00100000:2.0:1789760797.985474:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760797.985673:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950848 00000100:00100000:3.0:1789760797.985689:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950848 00000100:00100000:3.0:1789760797.985768:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728070468442 time 728089623632 state 0x0 ref 1 00000100:00100000:3.0:1789760797.985789:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728089884048 time 728089623632 state 0x1 ref 1 00000100:00100000:3.0:1789760797.985823:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728089884048 time 728110429753 state 0x0 ref 1 00000100:00100000:3.0:1789760797.985839:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc12c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503950848:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760797.985908:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81940480, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27140 00010000:00010000:3.0:1789760797.985920:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.985947:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760797.985953:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.985962:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.985994:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.993667:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987312 is committed 00002000:00010000:3.0:1789760797.993822:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81940480 -> 81944576 00002000:00010000:3.0:1789760797.993828:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160064 -> 160072 00010000:00010000:3.0:1789760797.993847:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.993863:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.993873:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.993881:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760797.993887:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.993897:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.993929:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.993937:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81940480->81944575] (req 81940480->81944575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.994097:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc12c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503950848:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8252us (8697us total) trans 4294987312 rc 0/0 00000100:00100000:3.0:1789760797.994111:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20016 00000100:00100000:3.0:1789760797.994125:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728089884048 time 728110429753 state 0x2 ref 0 00000100:00100000:1.0:1789760798.001880:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.004257:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503950976 00000100:00100000:0.0:1789760798.004278:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503950976 00000100:00100000:0.0:1789760798.004472:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728089884048 time 728110429753 state 0x0 ref 1 00000100:00100000:0.0:1789760798.004501:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728110690169 time 728110429753 state 0x1 ref 1 00000100:00100000:0.0:1789760798.004648:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728110690169 time 728129253183 state 0x0 ref 1 00000100:00100000:0.0:1789760798.004667:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503950976:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.004731:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81944576, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27148 00010000:00010000:0.0:1789760798.004742:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004762:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.004767:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.004774:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004790:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760798.005899:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760798.013854:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987313 is committed 00002000:00010000:0.0:1789760798.014582:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81944576 -> 81948672 00002000:00010000:0.0:1789760798.014591:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160072 -> 160080 00010000:00010000:0.0:1789760798.014608:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.014623:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.014634:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.014660:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.014666:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.014677:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.014703:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.014711:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81944576->81948671] (req 81944576->81948671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.014851:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503950976:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10181us (12978us total) trans 4294987313 rc 0/0 00000100:00100000:0.0:1789760798.014864:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20017 00000100:00100000:0.0:1789760798.014882:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728110690169 time 728129253183 state 0x2 ref 0 00000100:00100000:2.0:1789760798.022379:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.022677:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503951104 00000100:00100000:1.0:1789760798.022738:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503951104 00000100:00100000:1.0:1789760798.022929:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728110690169 time 728129253183 state 0x0 ref 1 00000100:00100000:1.0:1789760798.022948:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728129513599 time 728129253183 state 0x1 ref 1 00000100:00100000:1.0:1789760798.023036:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728129513599 time 728147640750 state 0x0 ref 1 00000100:00100000:1.0:1789760798.023051:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503951104:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760798.023119:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81948672, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27150 00010000:00010000:1.0:1789760798.023130:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.023151:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760798.023156:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.023177:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.023194:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.029564:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987314 is committed 00002000:00010000:1.0:1789760798.029764:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81948672 -> 81952768 00002000:00010000:1.0:1789760798.029770:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160080 -> 160088 00010000:00010000:1.0:1789760798.029788:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.029813:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.029828:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.029836:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.029840:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.029850:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.029876:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.029890:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000305527e/0xfa2da65b083f2b1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81948672->81952767] (req 81948672->81952767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.030060:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503951104:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7006us (7908us total) trans 4294987314 rc 0/0 00000100:00100000:1.0:1789760798.030073:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20018 00000100:00100000:1.0:1789760798.030085:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728129513599 time 728147640750 state 0x2 ref 0 00000100:00100000:2.0:1789760798.035113:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.035207:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503951232 00000100:00100000:1.0:1789760798.035219:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503951232 00000100:00100000:1.0:1789760798.035305:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728129513599 time 728147640750 state 0x0 ref 1 00000100:00100000:1.0:1789760798.035324:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728147901166 time 728147640750 state 0x1 ref 1 00000100:00100000:1.0:1789760798.035370:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728147901166 time 728159967378 state 0x0 ref 1 00000100:00100000:1.0:1789760798.035385:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503951232:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760798.035487:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81952768, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27158 00010000:00010000:1.0:1789760798.035500:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.035525:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760798.035531:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.035538:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.035558:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.041358:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987315 is committed 00002000:00010000:1.0:1789760798.047166:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81952768 -> 81956864 00002000:00010000:1.0:1789760798.047180:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160088 -> 160096 00010000:00010000:1.0:1789760798.047200:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.047217:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.047226:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.047235:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.047240:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.047251:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.047277:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.047286:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2b21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81952768->81956863] (req 81952768->81956863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.047482:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503951232:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12092us (12377us total) trans 4294987315 rc 0/0 00000100:00100000:1.0:1789760798.047495:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20019 00000100:00100000:1.0:1789760798.047508:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728147901166 time 728159967378 state 0x2 ref 0 00000100:00100000:2.0:1789760798.053218:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.053385:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503951360 00000100:00100000:3.0:1789760798.053431:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503951360 00000100:00100000:3.0:1789760798.053520:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728147901166 time 728159967378 state 0x0 ref 1 00000100:00100000:3.0:1789760798.053538:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728160227794 time 728159967378 state 0x1 ref 1 00000100:00100000:3.0:1789760798.053573:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728160227794 time 728178179166 state 0x0 ref 1 00000100:00100000:3.0:1789760798.053589:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503951360:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.053645:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81956864, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27160 00010000:00010000:3.0:1789760798.053654:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.053675:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760798.053679:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.053685:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.053699:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.064930:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987316 is committed 00002000:00010000:2.0:1789760798.065057:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81956864 -> 81960960 00002000:00010000:2.0:1789760798.065064:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160096 -> 160104 00010000:00010000:2.0:1789760798.065081:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.065097:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.065107:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.065116:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.065121:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.065131:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.065155:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.065228:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb07fe99/0xfa2da65b083f2b28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81956864->81960959] (req 81956864->81960959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.065488:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503951360:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11896us (12279us total) trans 4294987316 rc 0/0 00000100:00100000:2.0:1789760798.065500:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20020 00000100:00100000:2.0:1789760798.065574:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728160227794 time 728178179166 state 0x2 ref 0 00000100:00100000:2.0:1789760798.073566:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.073717:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503951488 00000100:00100000:0.0:1789760798.073732:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503951488 00000100:00100000:0.0:1789760798.073811:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728160227794 time 728178179166 state 0x0 ref 1 00000100:00100000:0.0:1789760798.073830:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728178439582 time 728178179166 state 0x1 ref 1 00000100:00100000:0.0:1789760798.073873:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728178439582 time 728198479091 state 0x0 ref 1 00000100:00100000:0.0:1789760798.073888:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951488:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.073966:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81960960, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27168 00010000:00010000:0.0:1789760798.073978:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.073999:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.074004:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.074012:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.074028:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.081200:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987317 is committed 00002000:00010000:2.0:1789760798.081369:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81960960 -> 81965056 00002000:00010000:2.0:1789760798.081376:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160104 -> 160112 00010000:00010000:2.0:1789760798.081413:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081428:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081437:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081453:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.081458:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081468:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081494:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081501:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2b2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81960960->81965055] (req 81960960->81965055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.081790:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951488:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7897us (8233us total) trans 4294987317 rc 0/0 00000100:00100000:2.0:1789760798.081804:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20021 00000100:00100000:2.0:1789760798.081816:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728178439582 time 728198479091 state 0x2 ref 0 00000100:00100000:1.0:1789760798.092733:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.092900:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503951616 00000100:00100000:2.0:1789760798.092915:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503951616 00000100:00100000:2.0:1789760798.092993:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728178439582 time 728198479091 state 0x0 ref 1 00000100:00100000:2.0:1789760798.093022:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728198739507 time 728198479091 state 0x1 ref 1 00000100:00100000:2.0:1789760798.093063:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728198739507 time 728217668840 state 0x0 ref 1 00000100:00100000:2.0:1789760798.093078:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951616:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.093153:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81965056, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27170 00010000:00010000:2.0:1789760798.093164:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.093183:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.093189:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.093196:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.093212:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.100350:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987318 is committed 00002000:00010000:2.0:1789760798.100957:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81965056 -> 81969152 00002000:00010000:2.0:1789760798.100965:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160112 -> 160120 00010000:00010000:2.0:1789760798.100983:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.101001:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.101011:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.101029:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.101035:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.101047:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.101073:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.101087:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81965056->81969151] (req 81965056->81969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.101382:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951616:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8298us (8654us total) trans 4294987318 rc 0/0 00000100:00100000:2.0:1789760798.101438:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20022 00000100:00100000:2.0:1789760798.101461:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728198739507 time 728217668840 state 0x2 ref 0 00000100:00100000:1.0:1789760798.109747:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.109873:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503951744 00000100:00100000:2.0:1789760798.109886:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503951744 00000100:00100000:2.0:1789760798.109963:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728198739507 time 728217668840 state 0x0 ref 1 00000100:00100000:2.0:1789760798.109982:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728217929256 time 728217668840 state 0x1 ref 1 00000100:00100000:2.0:1789760798.111091:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728217929256 time 728235688445 state 0x0 ref 1 00000100:00100000:2.0:1789760798.111124:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951744:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.111198:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81969152, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27178 00010000:00010000:2.0:1789760798.111210:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.111231:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.111236:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.111244:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.111269:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.122473:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987319 is committed 00002000:00010000:2.0:1789760798.122643:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81969152 -> 81973248 00002000:00010000:2.0:1789760798.122649:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160120 -> 160128 00010000:00010000:2.0:1789760798.122666:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.122682:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.122692:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.122700:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.122706:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.122717:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.122741:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.122750:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81969152->81973247] (req 81969152->81973247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.122866:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951744:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11747us (13138us total) trans 4294987319 rc 0/0 00000100:00100000:2.0:1789760798.122878:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20023 00000100:00100000:2.0:1789760798.122890:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728217929256 time 728235688445 state 0x2 ref 0 00000100:00100000:3.0:1789760798.126875:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.127587:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503951872 00000100:00100000:0.0:1789760798.127603:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503951872 00000100:00100000:0.0:1789760798.127685:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728217929256 time 728235688445 state 0x0 ref 1 00000100:00100000:0.0:1789760798.127705:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728235948861 time 728235688445 state 0x1 ref 1 00000100:00100000:0.0:1789760798.127732:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728235948861 time 728252338749 state 0x0 ref 1 00000100:00100000:0.0:1789760798.127745:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951872:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.127809:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81973248, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27180 00010000:00010000:0.0:1789760798.127823:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.127843:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.127849:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.127857:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.127875:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.138707:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987320 is committed 00002000:00010000:1.0:1789760798.138883:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81973248 -> 81977344 00002000:00010000:1.0:1789760798.138890:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160128 -> 160136 00010000:00010000:1.0:1789760798.138908:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.138933:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.138943:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.138951:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.138957:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.138967:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.139010:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.139019:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2b44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81973248->81977343] (req 81973248->81977343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.139157:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8945500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503951872:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11405us (12289us total) trans 4294987320 rc 0/0 00000100:00100000:1.0:1789760798.140050:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20024 00000100:00100000:1.0:1789760798.140066:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728235948861 time 728252338749 state 0x2 ref 0 00000100:00100000:1.0:1789760798.143458:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.143619:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952000 00000100:00100000:2.0:1789760798.143633:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952000 00000100:00100000:2.0:1789760798.143720:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728235948861 time 728252338749 state 0x0 ref 1 00000100:00100000:2.0:1789760798.143739:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728252599165 time 728252338749 state 0x1 ref 1 00000100:00100000:2.0:1789760798.143776:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728252599165 time 728268382014 state 0x0 ref 1 00000100:00100000:2.0:1789760798.143790:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503952000:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.143857:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81977344, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27188 00010000:00010000:2.0:1789760798.143869:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.143888:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.143894:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.143901:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.143928:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.149951:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.150033:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952128 00000100:00100000:2.0:1789760798.150046:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952128 00000100:00100000:2.0:1789760798.150128:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 496 00000100:00100000:2.0:1789760798.150141:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dd266c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503952128:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:2.0:1789760798.150249:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dd266c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:8b68caea-910d-4e7e-8105-d8da387b4d8a+13:2392:x1876699503952128:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 111us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760798.150258:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 496 00000100:00100000:1.0:1789760798.151702:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.151823:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952256 00000100:00100000:0.0:1789760798.151835:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952256 00000100:00100000:0.0:1789760798.151967:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 1001 00000100:00100000:0.0:1789760798.151980:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503952256:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760798.152073:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+8:2392:x1876699503952256:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 96us (382us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760798.152081:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 1001 00000100:00080000:1.0:1789760798.152134:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760798.152752:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987321 is committed 00002000:00010000:2.0:1789760798.152881:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81977344 -> 81981440 00002000:00010000:2.0:1789760798.152888:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160136 -> 160144 00010000:00010000:2.0:1789760798.152908:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.152936:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.152946:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.152954:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.152959:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.152970:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153006:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153014:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2b4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81977344->81981439] (req 81977344->81981439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.153104:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc687b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503952000:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9312us (9667us total) trans 4294987321 rc 0/0 00000100:00100000:2.0:1789760798.153116:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20025 00000100:00100000:2.0:1789760798.153128:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728252599165 time 728268382014 state 0x2 ref 0 00000100:00100000:1.0:1789760798.154615:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.154691:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.154841:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.155050:0:8237:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952384 00000100:00100000:0.0:1789760798.155063:0:8237:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952384 00000100:00100000:0.0:1789760798.155152:0:8237:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952512 00000100:00100000:0.0:1789760798.155156:0:8237:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952512 00000100:00100000:0.0:1789760798.155188:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 1002 00000100:00100000:0.0:1789760798.155199:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503952384:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760798.155296:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6879880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503952384:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 99us (694us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760798.155305:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 1002 00000100:00100000:0.0:1789760798.155324:0:8237:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 1003 00000100:00100000:0.0:1789760798.155331:0:8237:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503952512:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760798.155360:0:8237:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2393:x1876699503952512:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 30us (669us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760798.155366:0:8237:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 1003 00000100:00080000:2.0:1789760798.155561:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760798.155567:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760798.156021:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952640 00000100:00100000:0.0:1789760798.156027:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952640 00000100:00100000:0.0:1789760798.156082:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.58@tcp, seq: 1004 00000100:00100000:0.0:1789760798.156090:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503952640:12345-192.168.203.58@tcp:400:kworker.0 00000100:00100000:0.0:1789760798.156135:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6878e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:453b2994-2b6d-4a69-9aac-a9e10430c708+5:2392:x1876699503952640:12345-192.168.203.58@tcp:400:kworker.0 Request processed in 46us (1294us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760798.156142:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.58@tcp, seq: 1004 00000100:00080000:2.0:1789760798.156192:0:6508:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760798.166147:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.169031:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952768 00000100:00100000:1.0:1789760798.169054:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952768 00000100:00100000:1.0:1789760798.169152:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728252599165 time 728268382014 state 0x0 ref 1 00000100:00100000:1.0:1789760798.169580:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728268642430 time 728268382014 state 0x1 ref 1 00000100:00100000:1.0:1789760798.169688:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728268642430 time 728294293714 state 0x0 ref 1 00000100:00100000:1.0:1789760798.169709:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503952768:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760798.169776:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81981440, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27190 00010000:00010000:1.0:1789760798.169788:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.169807:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760798.169812:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.169820:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.169837:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.176558:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987322 is committed 00002000:00010000:1.0:1789760798.176769:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81981440 -> 81985536 00002000:00010000:1.0:1789760798.176776:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160144 -> 160152 00010000:00010000:1.0:1789760798.176794:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176821:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176832:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176841:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.176848:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176858:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176882:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176891:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032aaf5af/0xfa2da65b083f2b52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81981440->81985535] (req 81981440->81985535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.177135:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503952768:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7423us (10995us total) trans 4294987322 rc 0/0 00000100:00100000:1.0:1789760798.177148:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20026 00000100:00100000:1.0:1789760798.177160:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728268642430 time 728294293714 state 0x2 ref 0 00000100:00100000:1.0:1789760798.191001:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.191221:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503952896 00000100:00100000:3.0:1789760798.191235:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503952896 00000100:00100000:3.0:1789760798.191315:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728268642430 time 728294293714 state 0x0 ref 1 00000100:00100000:3.0:1789760798.191337:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728294554130 time 728294293714 state 0x1 ref 1 00000100:00100000:3.0:1789760798.191371:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728294554130 time 728315977761 state 0x0 ref 1 00000100:00100000:3.0:1789760798.191385:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503952896:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.191505:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81985536, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27198 00010000:00010000:3.0:1789760798.191516:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.191536:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760798.191542:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.191551:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.191568:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.207233:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987323 is committed 00002000:00010000:2.0:1789760798.209336:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81985536 -> 81989632 00002000:00010000:2.0:1789760798.209351:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160152 -> 160160 00010000:00010000:2.0:1789760798.209369:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.209384:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.209454:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.209468:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.209473:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.209491:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.209518:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.209525:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096231c44/0xfa2da65b083f2b59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81985536->81989631] (req 81985536->81989631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.209672:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503952896:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18281us (18680us total) trans 4294987323 rc 0/0 00000100:00100000:2.0:1789760798.209685:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20027 00000100:00100000:2.0:1789760798.209697:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728294554130 time 728315977761 state 0x2 ref 0 00000100:00100000:2.0:1789760798.213468:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.213699:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953024 00000100:00100000:0.0:1789760798.213711:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953024 00000100:00100000:0.0:1789760798.213785:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728294554130 time 728315977761 state 0x0 ref 1 00000100:00100000:0.0:1789760798.213804:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728316238177 time 728315977761 state 0x1 ref 1 00000100:00100000:0.0:1789760798.213829:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728316238177 time 728338436163 state 0x0 ref 1 00000100:00100000:0.0:1789760798.213843:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503953024:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.213906:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81989632, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271a0 00010000:00010000:0.0:1789760798.213917:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.213937:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.213943:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.213950:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.213966:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.225046:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987324 is committed 00002000:00010000:0.0:1789760798.228943:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81989632 -> 81993728 00002000:00010000:0.0:1789760798.228956:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160160 -> 160168 00010000:00010000:0.0:1789760798.228974:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.228988:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.228997:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.229006:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.229011:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.229027:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.229051:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.229059:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81989632->81993727] (req 81989632->81993727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.229196:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503953024:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15347us (15745us total) trans 4294987324 rc 0/0 00000100:00100000:0.0:1789760798.229209:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20028 00000100:00100000:0.0:1789760798.229221:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728316238177 time 728338436163 state 0x2 ref 0 00010000:00010000:2.0:1789760798.233146:0:6500:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760798.235618:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.235738:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953152 00000100:00100000:2.0:1789760798.235752:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953152 00000100:00100000:2.0:1789760798.235833:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728316238177 time 728338436163 state 0x0 ref 1 00000100:00100000:2.0:1789760798.235853:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728338696579 time 728338436163 state 0x1 ref 1 00000100:00100000:2.0:1789760798.235896:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728338696579 time 728360502539 state 0x0 ref 1 00000100:00100000:2.0:1789760798.235909:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503953152:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.235981:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81993728, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271a8 00010000:00010000:2.0:1789760798.235994:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.236013:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.236019:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.236026:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.236044:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.246706:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987325 is committed 00002000:00010000:2.0:1789760798.246896:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81993728 -> 81997824 00002000:00010000:2.0:1789760798.246903:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160168 -> 160176 00010000:00010000:2.0:1789760798.246921:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.246936:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.246945:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.246958:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.246964:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.246975:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.246999:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.247007:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81993728->81997823] (req 81993728->81997823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.247191:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503953152:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11275us (11581us total) trans 4294987325 rc 0/0 00000100:00100000:2.0:1789760798.247205:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20029 00000100:00100000:2.0:1789760798.247217:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728338696579 time 728360502539 state 0x2 ref 0 00000100:00100000:1.0:1789760798.252321:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.252635:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953280 00000100:00100000:2.0:1789760798.252650:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953280 00000100:00100000:2.0:1789760798.252727:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728338696579 time 728360502539 state 0x0 ref 1 00000100:00100000:2.0:1789760798.252746:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728360762955 time 728360502539 state 0x1 ref 1 00000100:00100000:2.0:1789760798.252823:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728360762955 time 728377428971 state 0x0 ref 1 00000100:00100000:2.0:1789760798.252840:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503953280:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.252916:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 81997824, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271b0 00010000:00010000:2.0:1789760798.252928:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.252947:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.252953:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.252960:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.252978:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.262209:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987326 is committed 00002000:00010000:2.0:1789760798.262704:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 81997824 -> 82001920 00002000:00010000:2.0:1789760798.262711:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160176 -> 160184 00010000:00010000:2.0:1789760798.262729:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.262745:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.262754:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.262762:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.262768:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.262778:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.262814:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.262824:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [81997824->82001919] (req 81997824->82001919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.262956:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503953280:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10110us (10644us total) trans 4294987326 rc 0/0 00000100:00100000:2.0:1789760798.262968:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20030 00000100:00100000:2.0:1789760798.262980:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728360762955 time 728377428971 state 0x2 ref 0 00000100:00100000:1.0:1789760798.273153:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.276077:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953408 00000100:00100000:2.0:1789760798.276099:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953408 00000100:00100000:2.0:1789760798.276201:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728360762955 time 728377428971 state 0x0 ref 1 00000100:00100000:2.0:1789760798.276219:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728377689387 time 728377428971 state 0x1 ref 1 00000100:00100000:2.0:1789760798.276375:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728377689387 time 728400979126 state 0x0 ref 1 00000100:00100000:2.0:1789760798.276412:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503953408:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.276561:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82001920, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271b8 00010000:00010000:2.0:1789760798.276574:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.276595:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.276600:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.276608:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.276625:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.285145:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987327 is committed 00002000:00010000:2.0:1789760798.285383:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82001920 -> 82006016 00002000:00010000:2.0:1789760798.285422:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160184 -> 160192 00010000:00010000:2.0:1789760798.285439:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.285465:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.285475:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.286886:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.286895:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.286912:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.286941:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.286951:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82001920->82006015] (req 82001920->82006015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.287274:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503953408:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10878us (14124us total) trans 4294987327 rc 0/0 00000100:00100000:3.0:1789760798.287288:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20031 00000100:00100000:3.0:1789760798.287301:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728377689387 time 728400979126 state 0x2 ref 0 00000100:00100000:1.0:1789760798.302912:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.303258:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953536 00000100:00100000:0.0:1789760798.303341:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953536 00000100:00100000:0.0:1789760798.303615:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728377689387 time 728400979126 state 0x0 ref 1 00000100:00100000:0.0:1789760798.303634:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728401239542 time 728400979126 state 0x1 ref 1 00000100:00100000:0.0:1789760798.303725:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728401239542 time 728428330213 state 0x0 ref 1 00000100:00100000:0.0:1789760798.303742:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503953536:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.303819:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82006016, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271c0 00010000:00010000:0.0:1789760798.303832:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.303853:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.303859:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.303867:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.303884:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.313213:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987328 is committed 00002000:00010000:0.0:1789760798.313809:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82006016 -> 82010112 00002000:00010000:0.0:1789760798.313817:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160192 -> 160200 00010000:00010000:0.0:1789760798.313834:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.313850:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.313860:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.313868:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.313874:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.313890:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.313915:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.313923:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82006016->82010111] (req 82006016->82010111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.314047:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503953536:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10302us (11143us total) trans 4294987328 rc 0/0 00000100:00100000:0.0:1789760798.314061:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20032 00000100:00100000:0.0:1789760798.314073:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728401239542 time 728428330213 state 0x2 ref 0 00000100:00100000:1.0:1789760798.321493:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.321541:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953664 00000100:00100000:2.0:1789760798.321556:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953664 00000100:00100000:2.0:1789760798.321629:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728401239542 time 728428330213 state 0x0 ref 1 00000100:00100000:2.0:1789760798.321660:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728428590629 time 728428330213 state 0x1 ref 1 00000100:00100000:2.0:1789760798.321696:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728428590629 time 728446302367 state 0x0 ref 1 00000100:00100000:2.0:1789760798.321714:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503953664:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.321777:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82010112, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271c8 00010000:00010000:2.0:1789760798.321788:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.321807:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.321812:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.321820:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.321836:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.336127:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987329 is committed 00002000:00010000:2.0:1789760798.336379:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82010112 -> 82014208 00002000:00010000:2.0:1789760798.336386:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160200 -> 160208 00010000:00010000:2.0:1789760798.336455:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.336470:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.336480:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.336490:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.336496:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.336506:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.336531:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.336540:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82010112->82014207] (req 82010112->82014207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.336835:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503953664:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15120us (15352us total) trans 4294987329 rc 0/0 00000100:00100000:2.0:1789760798.336847:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20033 00000100:00100000:2.0:1789760798.336859:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728428590629 time 728446302367 state 0x2 ref 0 00000100:00100000:1.0:1789760798.346295:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.346500:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953792 00000100:00100000:2.0:1789760798.346514:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953792 00000100:00100000:2.0:1789760798.346601:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728428590629 time 728446302367 state 0x0 ref 1 00000100:00100000:2.0:1789760798.346620:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728446562783 time 728446302367 state 0x1 ref 1 00000100:00100000:2.0:1789760798.346656:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728446562783 time 728471262351 state 0x0 ref 1 00000100:00100000:2.0:1789760798.346670:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503953792:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.346734:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82014208, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271d0 00010000:00010000:2.0:1789760798.346746:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.346767:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.346772:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.346779:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.346795:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.355246:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987330 is committed 00002000:00010000:1.0:1789760798.355701:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82014208 -> 82018304 00002000:00010000:1.0:1789760798.355707:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160208 -> 160216 00010000:00010000:1.0:1789760798.355723:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.355734:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.355741:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.355746:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.355750:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.355758:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.356064:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.356074:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82014208->82018303] (req 82014208->82018303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.356268:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc6ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503953792:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9593us (9980us total) trans 4294987330 rc 0/0 00000100:00100000:1.0:1789760798.356287:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20034 00000100:00100000:1.0:1789760798.356300:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728446562783 time 728471262351 state 0x2 ref 0 00000100:00100000:1.0:1789760798.365555:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.366066:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503953920 00000100:00100000:3.0:1789760798.366080:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503953920 00000100:00100000:3.0:1789760798.366158:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728446562783 time 728471262351 state 0x0 ref 1 00000100:00100000:3.0:1789760798.366189:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728471522767 time 728471262351 state 0x1 ref 1 00000100:00100000:3.0:1789760798.366223:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728471522767 time 728490829181 state 0x0 ref 1 00000100:00100000:3.0:1789760798.366238:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc933b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503953920:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.366300:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82018304, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271d8 00010000:00010000:3.0:1789760798.366311:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.366332:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760798.366337:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.366345:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.366364:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.377154:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987331 is committed 00002000:00010000:3.0:1789760798.377351:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82018304 -> 82022400 00002000:00010000:3.0:1789760798.377358:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160216 -> 160224 00010000:00010000:3.0:1789760798.377375:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.377427:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.377627:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.377696:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760798.377702:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.377713:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.377740:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.377749:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2b91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82018304->82022399] (req 82018304->82022399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.377941:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc933b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503953920:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11697us (12392us total) trans 4294987331 rc 0/0 00000100:00100000:3.0:1789760798.377954:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20035 00000100:00100000:3.0:1789760798.377967:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728471522767 time 728490829181 state 0x2 ref 0 00000100:00100000:2.0:1789760798.387116:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.387246:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954048 00000100:00100000:0.0:1789760798.387260:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954048 00000100:00100000:0.0:1789760798.387339:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728471522767 time 728490829181 state 0x0 ref 1 00000100:00100000:0.0:1789760798.387357:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728491089597 time 728490829181 state 0x1 ref 1 00000100:00100000:0.0:1789760798.387491:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728491089597 time 728512095450 state 0x0 ref 1 00000100:00100000:0.0:1789760798.387573:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954048:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.387643:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82022400, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271e0 00010000:00010000:0.0:1789760798.387655:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.387894:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.387901:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.387911:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.387931:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.396077:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987332 is committed 00002000:00010000:0.0:1789760798.398737:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82022400 -> 82026496 00002000:00010000:0.0:1789760798.398750:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160224 -> 160232 00010000:00010000:0.0:1789760798.398769:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.398786:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.398796:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.398805:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.398811:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.398822:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.398854:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.398863:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2b98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82022400->82026495] (req 82022400->82026495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.399073:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954048:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11498us (11966us total) trans 4294987332 rc 0/0 00000100:00100000:0.0:1789760798.399087:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20036 00000100:00100000:0.0:1789760798.399101:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728491089597 time 728512095450 state 0x2 ref 0 00000100:00100000:1.0:1789760798.404143:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.404719:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954176 00000100:00100000:2.0:1789760798.404735:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954176 00000100:00100000:2.0:1789760798.404820:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728491089597 time 728512095450 state 0x0 ref 1 00000100:00100000:2.0:1789760798.404840:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728512355866 time 728512095450 state 0x1 ref 1 00000100:00100000:2.0:1789760798.404880:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728512355866 time 728529486045 state 0x0 ref 1 00000100:00100000:2.0:1789760798.404893:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503954176:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.404969:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82026496, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271e8 00010000:00010000:2.0:1789760798.404981:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.405006:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.405012:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.405020:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.405038:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.415186:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987333 is committed 00002000:00010000:2.0:1789760798.415372:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82026496 -> 82030592 00002000:00010000:2.0:1789760798.415379:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160232 -> 160240 00010000:00010000:2.0:1789760798.415420:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.415453:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.415464:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.415473:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.415479:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.415499:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.415523:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.415530:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2b9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82026496->82030591] (req 82026496->82030591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.415667:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503954176:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10768us (11530us total) trans 4294987333 rc 0/0 00000100:00100000:2.0:1789760798.415680:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20037 00000100:00100000:2.0:1789760798.415693:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728512355866 time 728529486045 state 0x2 ref 0 00000100:00100000:1.0:1789760798.423869:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.424076:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954304 00000100:00100000:2.0:1789760798.424090:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954304 00000100:00100000:2.0:1789760798.424168:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728512355866 time 728529486045 state 0x0 ref 1 00000100:00100000:2.0:1789760798.424188:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728529746461 time 728529486045 state 0x1 ref 1 00000100:00100000:2.0:1789760798.424265:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728529746461 time 728548870560 state 0x0 ref 1 00000100:00100000:2.0:1789760798.424279:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954304:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.424351:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82030592, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271f0 00010000:00010000:2.0:1789760798.424363:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.424383:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.424407:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.424415:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.424433:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.432950:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987334 is committed 00002000:00010000:2.0:1789760798.433497:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82030592 -> 82034688 00002000:00010000:2.0:1789760798.433505:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160240 -> 160248 00010000:00010000:2.0:1789760798.433529:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.433546:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.433562:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.433571:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.433588:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.433600:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.433625:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.433634:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2ba6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82030592->82034687] (req 82030592->82034687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.433767:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de07fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954304:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9482us (9904us total) trans 4294987334 rc 0/0 00000100:00100000:2.0:1789760798.433780:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20038 00000100:00100000:2.0:1789760798.433793:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728529746461 time 728548870560 state 0x2 ref 0 00000100:00100000:1.0:1789760798.440351:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.440515:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954432 00000100:00100000:3.0:1789760798.440535:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954432 00000100:00100000:3.0:1789760798.440611:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728529746461 time 728548870560 state 0x0 ref 1 00000100:00100000:3.0:1789760798.440632:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728549130976 time 728548870560 state 0x1 ref 1 00000100:00100000:3.0:1789760798.440663:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728549130976 time 728565269899 state 0x0 ref 1 00000100:00100000:3.0:1789760798.440677:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff527b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503954432:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.440744:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82034688, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x271f8 00010000:00010000:3.0:1789760798.440755:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.440776:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760798.440781:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.440788:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.440811:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.449188:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987335 is committed 00002000:00010000:3.0:1789760798.449482:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82034688 -> 82038784 00002000:00010000:3.0:1789760798.449489:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160248 -> 160256 00010000:00010000:3.0:1789760798.449510:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.449526:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.449536:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.449544:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760798.449550:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.449561:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.449590:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.449598:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82034688->82038783] (req 82034688->82038783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.449740:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff527b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503954432:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9056us (9394us total) trans 4294987335 rc 0/0 00000100:00100000:3.0:1789760798.449754:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20039 00000100:00100000:3.0:1789760798.449767:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728549130976 time 728565269899 state 0x2 ref 0 00000100:00100000:1.0:1789760798.456347:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.461454:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954560 00000100:00100000:0.0:1789760798.461479:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954560 00000100:00100000:0.0:1789760798.461577:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728549130976 time 728565269899 state 0x0 ref 1 00000100:00100000:0.0:1789760798.461597:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728565530315 time 728565269899 state 0x1 ref 1 00000100:00100000:0.0:1789760798.461636:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728565530315 time 728586242246 state 0x0 ref 1 00000100:00100000:0.0:1789760798.461654:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954560:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.461728:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82038784, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27200 00010000:00010000:0.0:1789760798.461740:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.461760:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.461766:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.461774:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.461791:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.470833:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987336 is committed 00002000:00010000:0.0:1789760798.482572:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82038784 -> 82042880 00002000:00010000:0.0:1789760798.482587:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160256 -> 160264 00010000:00010000:0.0:1789760798.482607:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.482622:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.482632:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.482640:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.482646:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.482656:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.482682:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.482696:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2bb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82038784->82042879] (req 82038784->82042879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.482823:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954560:12345-192.168.203.58@tcp:4:dd.0 Request processed in 21166us (26496us total) trans 4294987336 rc 0/0 00000100:00100000:0.0:1789760798.482840:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20040 00000100:00100000:0.0:1789760798.482853:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728565530315 time 728586242246 state 0x2 ref 0 00000100:00100000:1.0:1789760798.488807:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.489450:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954688 00000100:00100000:2.0:1789760798.489466:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954688 00000100:00100000:2.0:1789760798.489544:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728565530315 time 728586242246 state 0x0 ref 1 00000100:00100000:2.0:1789760798.489562:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728586502662 time 728586242246 state 0x1 ref 1 00000100:00100000:2.0:1789760798.489600:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728586502662 time 728614205597 state 0x0 ref 1 00000100:00100000:2.0:1789760798.489615:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503954688:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.489678:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82042880, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27208 00010000:00010000:2.0:1789760798.489691:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.489712:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.489718:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.489726:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.489744:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.507335:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987337 is committed 00002000:00010000:2.0:1789760798.507497:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82042880 -> 82046976 00002000:00010000:2.0:1789760798.507504:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160264 -> 160272 00010000:00010000:2.0:1789760798.507532:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.507547:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.507557:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.507572:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.507578:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.507588:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.507613:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.507620:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82042880->82046975] (req 82042880->82046975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.507734:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff524000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503954688:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18115us (18936us total) trans 4294987337 rc 0/0 00000100:00100000:2.0:1789760798.507746:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20041 00000100:00100000:2.0:1789760798.507758:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728586502662 time 728614205597 state 0x2 ref 0 00000100:00100000:1.0:1789760798.517048:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.517590:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954816 00000100:00100000:2.0:1789760798.517603:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954816 00000100:00100000:2.0:1789760798.517678:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728586502662 time 728614205597 state 0x0 ref 1 00000100:00100000:2.0:1789760798.517698:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728614466013 time 728614205597 state 0x1 ref 1 00000100:00100000:2.0:1789760798.517736:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728614466013 time 728642341604 state 0x0 ref 1 00000100:00100000:2.0:1789760798.517754:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954816:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.517818:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82046976, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27210 00010000:00010000:2.0:1789760798.517829:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.517850:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.517855:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.517862:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.517878:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.525108:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987338 is committed 00002000:00010000:1.0:1789760798.525329:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82046976 -> 82051072 00002000:00010000:1.0:1789760798.525336:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160272 -> 160280 00010000:00010000:1.0:1789760798.525609:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.525626:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.525635:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.525644:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.525650:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.525662:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.525689:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.525698:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058e1e9f5/0xfa2da65b083f2bc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82046976->82051071] (req 82046976->82051071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.525864:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954816:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8105us (8825us total) trans 4294987338 rc 0/0 00000100:00100000:1.0:1789760798.525877:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20042 00000100:00100000:1.0:1789760798.525889:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728614466013 time 728642341604 state 0x2 ref 0 00000100:00100000:1.0:1789760798.533859:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.533959:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503954944 00000100:00100000:3.0:1789760798.533971:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503954944 00000100:00100000:3.0:1789760798.534046:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728614466013 time 728642341604 state 0x0 ref 1 00000100:00100000:3.0:1789760798.534066:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728642602020 time 728642341604 state 0x1 ref 1 00000100:00100000:3.0:1789760798.534106:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728642602020 time 728658712129 state 0x0 ref 1 00000100:00100000:3.0:1789760798.534119:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954944:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.534192:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82051072, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27218 00010000:00010000:3.0:1789760798.534206:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.534226:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760798.534231:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.534238:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.534256:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.540667:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987339 is committed 00002000:00010000:3.0:1789760798.540924:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82051072 -> 82055168 00002000:00010000:3.0:1789760798.540931:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160280 -> 160288 00010000:00010000:3.0:1789760798.540949:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.540965:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.541032:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.541041:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760798.541047:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.541058:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.541083:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.541092:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2bc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82051072->82055167] (req 82051072->82055167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.541257:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dff525880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503954944:12345-192.168.203.58@tcp:4:dd.0 Request processed in 7132us (7404us total) trans 4294987339 rc 0/0 00000100:00100000:3.0:1789760798.541271:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20043 00000100:00100000:3.0:1789760798.541284:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728642602020 time 728658712129 state 0x2 ref 0 00000100:00100000:0.0:1789760798.549770:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.550110:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955072 00000100:00100000:1.0:1789760798.550125:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955072 00000100:00100000:1.0:1789760798.550212:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728642602020 time 728658712129 state 0x0 ref 1 00000100:00100000:1.0:1789760798.550231:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728658972545 time 728658712129 state 0x1 ref 1 00000100:00100000:1.0:1789760798.550311:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728658972545 time 728674917255 state 0x0 ref 1 00000100:00100000:1.0:1789760798.550326:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955072:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760798.550453:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82055168, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27220 00010000:00010000:1.0:1789760798.550465:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.550493:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760798.550499:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.550507:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.550524:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.556815:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987340 is committed 00002000:00010000:0.0:1789760798.563738:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82055168 -> 82059264 00002000:00010000:0.0:1789760798.563750:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160288 -> 160296 00010000:00010000:0.0:1789760798.563768:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.563783:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.563792:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.563890:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1800 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.563896:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.563907:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.563937:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.563946:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b065d52f/0xfa2da65b083f2bd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82055168->82059263] (req 82055168->82059263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.564120:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955072:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13788us (14358us total) trans 4294987340 rc 0/0 00000100:00100000:0.0:1789760798.564131:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20044 00000100:00100000:0.0:1789760798.564144:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728658972545 time 728674917255 state 0x2 ref 0 00000100:00100000:1.0:1789760798.573871:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.574910:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955200 00000100:00100000:2.0:1789760798.574927:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955200 00000100:00100000:2.0:1789760798.575016:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728658972545 time 728674917255 state 0x0 ref 1 00000100:00100000:2.0:1789760798.575035:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728675177671 time 728674917255 state 0x1 ref 1 00000100:00100000:2.0:1789760798.575068:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728675177671 time 728699673842 state 0x0 ref 1 00000100:00100000:2.0:1789760798.575083:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce85a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503955200:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.575149:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82059264, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27228 00010000:00010000:2.0:1789760798.575161:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.575304:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.575309:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.575317:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.575335:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.587927:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987341 is committed 00002000:00010000:2.0:1789760798.588125:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82059264 -> 82063360 00002000:00010000:2.0:1789760798.588133:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160296 -> 160304 00010000:00010000:2.0:1789760798.588151:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.588208:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.588219:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.588229:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.588235:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.588246:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.588279:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.588288:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82059264->82063359] (req 82059264->82063359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.588615:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce85a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503955200:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13527us (14751us total) trans 4294987341 rc 0/0 00000100:00100000:2.0:1789760798.588628:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20045 00000100:00100000:2.0:1789760798.588642:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728675177671 time 728699673842 state 0x2 ref 0 00000100:00100000:1.0:1789760798.596215:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.597570:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955328 00000100:00100000:3.0:1789760798.597588:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955328 00000100:00100000:0.0:1789760798.597754:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728675177671 time 728699673842 state 0x0 ref 1 00000100:00100000:0.0:1789760798.597775:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728699934258 time 728699673842 state 0x1 ref 1 00000100:00100000:0.0:1789760798.597814:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728699934258 time 728722420128 state 0x0 ref 1 00000100:00100000:0.0:1789760798.597830:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cce858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503955328:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.597895:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82063360, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27230 00010000:00010000:0.0:1789760798.597906:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.597927:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.597932:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.597941:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.597959:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.608746:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987342 is committed 00002000:00010000:0.0:1789760798.609158:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82063360 -> 82067456 00002000:00010000:0.0:1789760798.609171:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160304 -> 160312 00010000:00010000:0.0:1789760798.609188:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.609203:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.609214:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.609221:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dbc00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.609227:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.609237:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.609260:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.609273:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7cb487b/0xfa2da65b083f2bde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82063360->82067455] (req 82063360->82067455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.609448:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cce858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503955328:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11564us (13190us total) trans 4294987342 rc 0/0 00000100:00100000:0.0:1789760798.609461:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20046 00000100:00100000:0.0:1789760798.609474:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728699934258 time 728722420128 state 0x2 ref 0 00000100:00100000:0.0:1789760798.614534:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.614605:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955456 00000100:00100000:3.0:1789760798.614619:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955456 00000100:00100000:3.0:1789760798.614691:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728699934258 time 728722420128 state 0x0 ref 1 00000100:00100000:3.0:1789760798.614710:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728722680544 time 728722420128 state 0x1 ref 1 00000100:00100000:3.0:1789760798.614754:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728722680544 time 728739351599 state 0x0 ref 1 00000100:00100000:3.0:1789760798.614769:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503955456:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.614846:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82067456, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27238 00010000:00010000:3.0:1789760798.614860:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.614881:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760798.614893:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.614901:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.614920:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.624776:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987343 is committed 00002000:00010000:3.0:1789760798.624900:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82067456 -> 82071552 00002000:00010000:3.0:1789760798.624908:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160312 -> 160320 00010000:00010000:3.0:1789760798.624924:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.624938:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.624948:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.624967:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760798.624973:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.624984:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.625010:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.625018:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2be5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82067456->82071551] (req 82067456->82071551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.625147:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503955456:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10374us (10622us total) trans 4294987343 rc 0/0 00000100:00100000:3.0:1789760798.625160:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20047 00000100:00100000:3.0:1789760798.625172:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728722680544 time 728739351599 state 0x2 ref 0 00000100:00100000:1.0:1789760798.630149:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.630273:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955584 00000100:00100000:0.0:1789760798.630287:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955584 00000100:00100000:0.0:1789760798.630367:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728722680544 time 728739351599 state 0x0 ref 1 00000100:00100000:0.0:1789760798.630386:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728739612015 time 728739351599 state 0x1 ref 1 00000100:00100000:0.0:1789760798.630456:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728739612015 time 728755061733 state 0x0 ref 1 00000100:00100000:0.0:1789760798.630472:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955584:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.630532:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82071552, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27240 00010000:00010000:0.0:1789760798.630542:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.630560:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.630565:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.630571:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.630587:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.635965:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987344 is committed 00002000:00010000:1.0:1789760798.636749:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82071552 -> 82075648 00002000:00010000:1.0:1789760798.636757:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160320 -> 160328 00010000:00010000:1.0:1789760798.636775:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.636792:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.636802:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.636816:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.636821:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.636833:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.636859:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.636867:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2bec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82071552->82075647] (req 82071552->82075647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.637138:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955584:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6660us (7073us total) trans 4294987344 rc 0/0 00000100:00100000:1.0:1789760798.637152:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20048 00000100:00100000:1.0:1789760798.637165:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728739612015 time 728755061733 state 0x2 ref 0 00000100:00100000:0.0:1789760798.643503:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.643740:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955712 00000100:00100000:2.0:1789760798.643753:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955712 00000100:00100000:2.0:1789760798.643838:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728739612015 time 728755061733 state 0x0 ref 1 00000100:00100000:2.0:1789760798.643862:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728755322149 time 728755061733 state 0x1 ref 1 00000100:00100000:2.0:1789760798.643911:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728755322149 time 728768506364 state 0x0 ref 1 00000100:00100000:2.0:1789760798.643927:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955712:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.643998:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82075648, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27248 00010000:00010000:2.0:1789760798.644008:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.644031:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.644036:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.644042:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.644058:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.652274:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987345 is committed 00002000:00010000:2.0:1789760798.653023:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82075648 -> 82079744 00002000:00010000:2.0:1789760798.653034:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160328 -> 160336 00010000:00010000:2.0:1789760798.653053:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.653069:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.653080:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.653089:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3000 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.653095:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.653104:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.653133:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.653141:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd272b2d/0xfa2da65b083f2bf3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82075648->82079743] (req 82075648->82079743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.653272:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8feaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955712:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9342us (9788us total) trans 4294987345 rc 0/0 00000100:00100000:2.0:1789760798.653286:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20049 00000100:00100000:2.0:1789760798.653299:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728755322149 time 728768506364 state 0x2 ref 0 00000100:00100000:0.0:1789760798.660585:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.660678:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955840 00000100:00100000:1.0:1789760798.660691:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955840 00000100:00100000:1.0:1789760798.660766:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728755322149 time 728768506364 state 0x0 ref 1 00000100:00100000:1.0:1789760798.660784:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728768766780 time 728768506364 state 0x1 ref 1 00000100:00100000:1.0:1789760798.660867:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728768766780 time 728785472328 state 0x0 ref 1 00000100:00100000:1.0:1789760798.660885:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503955840:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760798.660951:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82079744, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27250 00010000:00010000:1.0:1789760798.660962:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.660982:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760798.660988:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.660995:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.661012:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.674242:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987346 is committed 00002000:00010000:0.0:1789760798.674539:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82079744 -> 82083840 00002000:00010000:0.0:1789760798.674546:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160336 -> 160344 00010000:00010000:0.0:1789760798.674564:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.674579:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.674589:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.674596:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.674602:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.674613:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.674640:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.674649:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f63f2e9b/0xfa2da65b083f2bfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82079744->82083839] (req 82079744->82083839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.674782:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503955840:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13892us (14207us total) trans 4294987346 rc 0/0 00000100:00100000:0.0:1789760798.674795:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20050 00000100:00100000:0.0:1789760798.674809:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728768766780 time 728785472328 state 0x2 ref 0 00000100:00100000:1.0:1789760798.681352:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.681505:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503955968 00000100:00100000:3.0:1789760798.681518:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503955968 00000100:00100000:3.0:1789760798.681640:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728768766780 time 728785472328 state 0x0 ref 1 00000100:00100000:3.0:1789760798.681661:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728785732744 time 728785472328 state 0x1 ref 1 00000100:00100000:3.0:1789760798.681703:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728785732744 time 728806308449 state 0x0 ref 1 00000100:00100000:3.0:1789760798.681717:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955968:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.681782:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82083840, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27258 00010000:00010000:3.0:1789760798.681794:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.681816:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760798.681821:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.681829:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.681846:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.693011:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987347 is committed 00002000:00010000:2.0:1789760798.693224:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82083840 -> 82087936 00002000:00010000:2.0:1789760798.693230:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160344 -> 160352 00010000:00010000:2.0:1789760798.693249:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.693265:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.693276:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.693283:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.693289:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.693305:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.693330:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.693339:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82083840->82087935] (req 82083840->82087935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.693588:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503955968:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11866us (12242us total) trans 4294987347 rc 0/0 00000100:00100000:2.0:1789760798.693600:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20051 00000100:00100000:2.0:1789760798.693613:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728785732744 time 728806308449 state 0x2 ref 0 00000100:00100000:2.0:1789760798.701806:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.701920:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956096 00000100:00100000:0.0:1789760798.701933:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956096 00000100:00100000:0.0:1789760798.702082:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728785732744 time 728806308449 state 0x0 ref 1 00000100:00100000:0.0:1789760798.702101:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728806568865 time 728806308449 state 0x1 ref 1 00000100:00100000:0.0:1789760798.702194:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728806568865 time 728826798694 state 0x0 ref 1 00000100:00100000:0.0:1789760798.702208:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956096:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.702286:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82087936, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27260 00010000:00010000:0.0:1789760798.702298:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702319:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.702325:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.702333:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702357:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.715168:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987348 is committed 00002000:00010000:0.0:1789760798.715587:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82087936 -> 82092032 00002000:00010000:0.0:1789760798.715595:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160352 -> 160360 00010000:00010000:0.0:1789760798.715614:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.715630:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.715640:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.715647:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.715653:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.715663:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.715688:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.715696:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82087936->82092031] (req 82087936->82092031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.715877:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956096:12345-192.168.203.58@tcp:4:dd.0 Request processed in 13662us (14079us total) trans 4294987348 rc 0/0 00000100:00100000:0.0:1789760798.715891:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20052 00000100:00100000:0.0:1789760798.715903:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728806568865 time 728826798694 state 0x2 ref 0 00000100:00100000:0.0:1789760798.723130:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.723231:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956224 00000100:00100000:2.0:1789760798.723244:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956224 00000100:00100000:2.0:1789760798.723330:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728806568865 time 728826798694 state 0x0 ref 1 00000100:00100000:2.0:1789760798.723348:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728827059110 time 728826798694 state 0x1 ref 1 00000100:00100000:2.0:1789760798.723422:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728827059110 time 728847993430 state 0x0 ref 1 00000100:00100000:2.0:1789760798.723438:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956224:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.723517:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82092032, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27268 00010000:00010000:2.0:1789760798.723529:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.723549:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.723554:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.723562:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.723579:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.734955:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987349 is committed 00002000:00010000:2.0:1789760798.735348:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82092032 -> 82096128 00002000:00010000:2.0:1789760798.735357:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160360 -> 160368 00010000:00010000:2.0:1789760798.735386:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.735432:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.735451:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.735459:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.735465:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.735480:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.735499:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.735506:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca634b48/0xfa2da65b083f2c0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82092032->82096127] (req 82092032->82096127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.735593:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc8fe9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956224:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12153us (12473us total) trans 4294987349 rc 0/0 00000100:00100000:2.0:1789760798.735603:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20053 00000100:00100000:2.0:1789760798.735613:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728827059110 time 728847993430 state 0x2 ref 0 00000100:00100000:1.0:1789760798.740714:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.741106:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956352 00000100:00100000:0.0:1789760798.741120:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956352 00000100:00100000:0.0:1789760798.741196:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728827059110 time 728847993430 state 0x0 ref 1 00000100:00100000:0.0:1789760798.741226:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728848253846 time 728847993430 state 0x1 ref 1 00000100:00100000:0.0:1789760798.741438:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728848253846 time 728865870785 state 0x0 ref 1 00000100:00100000:0.0:1789760798.741465:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503956352:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.741531:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82096128, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27270 00010000:00010000:0.0:1789760798.741543:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.741563:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.741568:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.741575:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.741593:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.750225:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987350 is committed 00002000:00010000:0.0:1789760798.750383:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82096128 -> 82100224 00002000:00010000:0.0:1789760798.750447:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160368 -> 160376 00010000:00010000:0.0:1789760798.750466:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.750491:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.750501:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.750508:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.750514:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.750524:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.750549:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.750557:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82096128->82100223] (req 82096128->82100223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.750686:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503956352:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9217us (9977us total) trans 4294987350 rc 0/0 00000100:00100000:0.0:1789760798.750698:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20054 00000100:00100000:0.0:1789760798.750711:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728848253846 time 728865870785 state 0x2 ref 0 00000100:00100000:0.0:1789760798.758252:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.759551:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956480 00000100:00100000:1.0:1789760798.759571:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956480 00000100:00100000:1.0:1789760798.759652:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728848253846 time 728865870785 state 0x0 ref 1 00000100:00100000:1.0:1789760798.759671:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728866131201 time 728865870785 state 0x1 ref 1 00000100:00100000:1.0:1789760798.759709:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728866131201 time 728884314755 state 0x0 ref 1 00000100:00100000:1.0:1789760798.759724:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0775180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956480:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760798.759794:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82100224, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27278 00010000:00010000:1.0:1789760798.759804:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.759823:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760798.759829:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.759836:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.759852:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.767687:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987351 is committed 00002000:00010000:2.0:1789760798.768646:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82100224 -> 82104320 00002000:00010000:2.0:1789760798.768656:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160376 -> 160384 00010000:00010000:2.0:1789760798.768675:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.768691:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.768701:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.768715:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.768721:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.768731:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.768757:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.768766:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1afb8b9/0xfa2da65b083f2c1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82100224->82104319] (req 82100224->82104319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.769373:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0775180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956480:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9642us (11126us total) trans 4294987351 rc 0/0 00000100:00100000:2.0:1789760798.769459:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20055 00000100:00100000:2.0:1789760798.769473:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728866131201 time 728884314755 state 0x2 ref 0 00000100:00100000:1.0:1789760798.789302:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.789491:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956608 00000100:00100000:0.0:1789760798.789504:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956608 00000100:00100000:0.0:1789760798.789578:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728866131201 time 728884314755 state 0x0 ref 1 00000100:00100000:0.0:1789760798.789597:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728884575171 time 728884314755 state 0x1 ref 1 00000100:00100000:0.0:1789760798.789629:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728884575171 time 728914235464 state 0x0 ref 1 00000100:00100000:0.0:1789760798.789643:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503956608:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.789706:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82104320, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x27280 00010000:00010000:0.0:1789760798.789717:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.789737:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.789742:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.789750:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.789766:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.799213:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987352 is committed 00002000:00010000:0.0:1789760798.799357:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82104320 -> 82108416 00002000:00010000:0.0:1789760798.799363:0:8435:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760797 -> 1789760798 00002000:00010000:0.0:1789760798.799366:0:8435:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760797 -> 1789760798 00002000:00010000:0.0:1789760798.799368:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160384 -> 160392 00010000:00010000:0.0:1789760798.799384:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.799435:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.799459:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.799467:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dae00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.799473:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.799483:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.799508:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.799516:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d815904/0xfa2da65b083f2c24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82104320->82108415] (req 82104320->82108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.799685:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2393:x1876699503956608:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10038us (10391us total) trans 4294987352 rc 0/0 00000100:00100000:0.0:1789760798.799702:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20056 00000100:00100000:0.0:1789760798.799714:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728884575171 time 728914235464 state 0x2 ref 0 00000100:00100000:1.0:1789760798.805527:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.805685:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956736 00000100:00100000:2.0:1789760798.805697:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956736 00000100:00100000:2.0:1789760798.805774:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728884575171 time 728914235464 state 0x0 ref 1 00000100:00100000:2.0:1789760798.805794:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728914495880 time 728914235464 state 0x1 ref 1 00000100:00100000:2.0:1789760798.805834:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728914495880 time 728930440618 state 0x0 ref 1 00000100:00100000:2.0:1789760798.805851:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503956736:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.805912:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82108416, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27288 00010000:00010000:2.0:1789760798.805923:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.805943:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.805948:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.805956:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.805982:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.812038:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987353 is committed 00002000:00010000:1.0:1789760798.812265:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82108416 -> 82112512 00002000:00010000:1.0:1789760798.812271:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160392 -> 160400 00010000:00010000:1.0:1789760798.812288:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.812304:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.812313:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.812322:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.812328:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.812339:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.812365:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.812372:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a300d19f/0xfa2da65b083f2c2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82108416->82112511] (req 82108416->82112511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.812535:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503956736:12345-192.168.203.58@tcp:4:dd.0 Request processed in 6677us (7015us total) trans 4294987353 rc 0/0 00000100:00100000:1.0:1789760798.812547:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20057 00000100:00100000:1.0:1789760798.812560:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728914495880 time 728930440618 state 0x2 ref 0 00000100:00100000:2.0:1789760798.823944:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760798.824316:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956864 00000100:00100000:1.0:1789760798.824330:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956864 00000100:00100000:1.0:1789760798.824452:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728914495880 time 728930440618 state 0x0 ref 1 00000100:00100000:1.0:1789760798.824472:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728930701034 time 728930440618 state 0x1 ref 1 00000100:00100000:1.0:1789760798.824514:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728930701034 time 728949120740 state 0x0 ref 1 00000100:00100000:1.0:1789760798.824528:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503956864:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760798.824591:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82112512, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27290 00010000:00010000:1.0:1789760798.824603:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.824624:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760798.824628:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.824636:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.824651:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.832715:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987354 is committed 00002000:00010000:0.0:1789760798.832849:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82112512 -> 82116608 00002000:00010000:0.0:1789760798.832856:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160400 -> 160408 00010000:00010000:0.0:1789760798.832874:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.832890:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.832901:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.832909:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760798.832916:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.832926:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.832951:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.832961:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4ccde51/0xfa2da65b083f2c32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82112512->82116607] (req 82112512->82116607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.833085:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503956864:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8553us (9150us total) trans 4294987354 rc 0/0 00000100:00100000:0.0:1789760798.833098:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20058 00000100:00100000:0.0:1789760798.833110:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728930701034 time 728949120740 state 0x2 ref 0 00000100:00100000:0.0:1789760798.840158:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.840475:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503956992 00000100:00100000:2.0:1789760798.840490:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503956992 00000100:00100000:2.0:1789760798.840963:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728930701034 time 728949120740 state 0x0 ref 1 00000100:00100000:2.0:1789760798.840983:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728949381156 time 728949120740 state 0x1 ref 1 00000100:00100000:2.0:1789760798.841026:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728949381156 time 728965631816 state 0x0 ref 1 00000100:00100000:2.0:1789760798.841043:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956992:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.841122:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82116608, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27298 00010000:00010000:2.0:1789760798.841134:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.841153:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760798.841158:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.841165:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.841180:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.849737:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987355 is committed 00002000:00010000:2.0:1789760798.849900:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82116608 -> 82120704 00002000:00010000:2.0:1789760798.849907:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160408 -> 160416 00010000:00010000:2.0:1789760798.849923:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.849938:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.849948:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.849956:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.849961:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.849972:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.849999:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.850009:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82116608->82120703] (req 82116608->82120703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.850197:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0774380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503956992:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9150us (10046us total) trans 4294987355 rc 0/0 00000100:00100000:2.0:1789760798.850218:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20059 00000100:00100000:2.0:1789760798.850231:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728949381156 time 728965631816 state 0x2 ref 0 00000100:00100000:1.0:1789760798.859939:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.860757:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503957120 00000100:00100000:0.0:1789760798.860773:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503957120 00000100:00100000:0.0:1789760798.860847:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728949381156 time 728965631816 state 0x0 ref 1 00000100:00100000:0.0:1789760798.860866:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728965892232 time 728965631816 state 0x1 ref 1 00000100:00100000:0.0:1789760798.861043:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728965892232 time 728985648014 state 0x0 ref 1 00000100:00100000:0.0:1789760798.861059:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503957120:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.861126:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82120704, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272a0 00010000:00010000:0.0:1789760798.861137:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.861157:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760798.861162:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.861170:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.861187:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.868830:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987356 is committed 00002000:00010000:3.0:1789760798.869374:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82120704 -> 82124800 00002000:00010000:3.0:1789760798.869382:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160416 -> 160424 00010000:00010000:3.0:1789760798.869425:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.869449:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.869460:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.869468:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3daa00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760798.869473:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.869555:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.869583:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.869592:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d64ce3b/0xfa2da65b083f2c40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82120704->82124799] (req 82120704->82124799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.869775:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503957120:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8712us (9846us total) trans 4294987356 rc 0/0 00000100:00100000:3.0:1789760798.869792:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20060 00000100:00100000:3.0:1789760798.869804:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728965892232 time 728985648014 state 0x2 ref 0 00000100:00100000:1.0:1789760798.882561:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.882674:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503957248 00000100:00100000:2.0:1789760798.882687:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503957248 00000100:00100000:2.0:1789760798.882762:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728965892232 time 728985648014 state 0x0 ref 1 00000100:00100000:2.0:1789760798.882780:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728985908430 time 728985648014 state 0x1 ref 1 00000100:00100000:2.0:1789760798.882819:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728985908430 time 729007424569 state 0x0 ref 1 00000100:00100000:2.0:1789760798.882832:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503957248:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.882896:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82124800, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272a8 00010000:00010000:2.0:1789760798.882907:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.882925:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760798.882940:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.882949:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.882969:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.892141:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987357 is committed 00002000:00010000:2.0:1789760798.892745:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82124800 -> 82128896 00002000:00010000:2.0:1789760798.892753:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160424 -> 160432 00010000:00010000:2.0:1789760798.892769:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.892784:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.892794:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.892802:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.892808:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.892818:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.892843:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.892859:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82124800->82128895] (req 82124800->82128895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.893027:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc876df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503957248:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10190us (10474us total) trans 4294987357 rc 0/0 00000100:00100000:2.0:1789760798.893040:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20061 00000100:00100000:2.0:1789760798.893052:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728985908430 time 729007424569 state 0x2 ref 0 00000100:00100000:1.0:1789760798.899080:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760798.899369:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503957376 00000100:00100000:0.0:1789760798.899382:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503957376 00000100:00100000:0.0:1789760798.899811:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 728985908430 time 729007424569 state 0x0 ref 1 00000100:00100000:0.0:1789760798.899830:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729007684985 time 729007424569 state 0x1 ref 1 00000100:00100000:0.0:1789760798.899865:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729007684985 time 729024471248 state 0x0 ref 1 00000100:00100000:0.0:1789760798.899880:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503957376:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760798.901059:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82128896, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272b0 00010000:00010000:0.0:1789760798.901077:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.901098:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760798.901103:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.901110:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.901126:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.908806:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987358 is committed 00002000:00010000:1.0:1789760798.910705:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82128896 -> 82132992 00002000:00010000:1.0:1789760798.910719:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160432 -> 160440 00010000:00010000:1.0:1789760798.910737:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.910753:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.910763:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.910771:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da200 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.910777:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.910787:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.910812:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.910820:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b36af1d8/0xfa2da65b083f2c4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82128896->82132991] (req 82128896->82132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.911633:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503957376:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11743us (12555us total) trans 4294987358 rc 0/0 00000100:00100000:1.0:1789760798.911649:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20062 00000100:00100000:1.0:1789760798.911661:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729007684985 time 729024471248 state 0x2 ref 0 00000100:00100000:0.0:1789760798.924338:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.924451:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503957504 00000100:00100000:2.0:1789760798.924476:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503957504 00000100:00100000:2.0:1789760798.924554:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729007684985 time 729024471248 state 0x0 ref 1 00000100:00100000:2.0:1789760798.924573:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729024731664 time 729024471248 state 0x1 ref 1 00000100:00100000:2.0:1789760798.924620:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729024731664 time 729049225958 state 0x0 ref 1 00000100:00100000:2.0:1789760798.924635:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503957504:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.924701:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82132992, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272b8 00010000:00010000:2.0:1789760798.924714:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.924737:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760798.924742:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.924750:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.924782:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.932966:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987359 is committed 00002000:00010000:2.0:1789760798.933120:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82132992 -> 82137088 00002000:00010000:2.0:1789760798.933126:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160440 -> 160448 00010000:00010000:2.0:1789760798.933143:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.933159:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.933180:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.933188:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760798.933195:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.933206:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.933232:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.933242:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82132992->82137087] (req 82132992->82137087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.933370:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5de0776a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503957504:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8730us (9041us total) trans 4294987359 rc 0/0 00000100:00100000:2.0:1789760798.933381:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20063 00000100:00100000:2.0:1789760798.934637:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729024731664 time 729049225958 state 0x2 ref 0 00000100:00100000:0.0:1789760798.942982:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760798.943084:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503957632 00000100:00100000:3.0:1789760798.943098:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503957632 00000100:00100000:3.0:1789760798.943171:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729024731664 time 729049225958 state 0x0 ref 1 00000100:00100000:3.0:1789760798.943191:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729049486374 time 729049225958 state 0x1 ref 1 00000100:00100000:3.0:1789760798.943236:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729049486374 time 729067834164 state 0x0 ref 1 00000100:00100000:3.0:1789760798.943250:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503957632:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760798.943324:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82137088, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272c0 00010000:00010000:3.0:1789760798.943336:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.943356:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760798.943361:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.943369:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.943385:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.953035:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987360 is committed 00002000:00010000:3.0:1789760798.953819:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82137088 -> 82141184 00002000:00010000:3.0:1789760798.953826:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160448 -> 160456 00010000:00010000:3.0:1789760798.953846:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.953863:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.953873:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.953881:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760798.953886:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.953896:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.953934:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.953942:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82137088->82141183] (req 82137088->82141183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.954076:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503957632:12345-192.168.203.58@tcp:4:dd.0 Request processed in 10820us (11101us total) trans 4294987360 rc 0/0 00000100:00100000:3.0:1789760798.954088:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20064 00000100:00100000:3.0:1789760798.954101:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729049486374 time 729067834164 state 0x2 ref 0 00000100:00100000:1.0:1789760798.968895:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:2.0:1789760798.970221:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503957760 00000100:00100000:2.0:1789760798.970240:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503957760 00000100:00100000:2.0:1789760798.970333:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729049486374 time 729067834164 state 0x0 ref 1 00000100:00100000:2.0:1789760798.970354:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729068094580 time 729067834164 state 0x1 ref 1 00000100:00100000:2.0:1789760798.970420:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729068094580 time 729095024817 state 0x0 ref 1 00000100:00100000:2.0:1789760798.971352:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503957760:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760798.971498:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82141184, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272c8 00010000:00010000:2.0:1789760798.971518:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.971539:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760798.971545:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.971553:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.971568:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.989571:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987361 is committed 00002000:00010000:1.0:1789760798.990042:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82141184 -> 82145280 00002000:00010000:1.0:1789760798.990047:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160456 -> 160464 00010000:00010000:1.0:1789760798.990060:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.990074:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.990090:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.990097:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f3a00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760798.990101:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.990111:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.990130:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.990136:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed4579e/0xfa2da65b083f2c63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82141184->82145279] (req 82141184->82145279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.990248:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503957760:12345-192.168.203.58@tcp:4:dd.0 Request processed in 19812us (21364us total) trans 4294987361 rc 0/0 00000100:00100000:1.0:1789760798.990258:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20065 00000100:00100000:1.0:1789760798.990268:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729068094580 time 729095024817 state 0x2 ref 0 00000100:00100000:0.0:1789760799.001182:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760799.005084:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503957888 00000100:00100000:1.0:1789760799.005109:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503957888 00000100:00100000:0.0:1789760799.005575:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729068094580 time 729095024817 state 0x0 ref 1 00000100:00100000:0.0:1789760799.005596:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729095285233 time 729095024817 state 0x1 ref 1 00000100:00100000:0.0:1789760799.005778:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729095285233 time 729130382390 state 0x0 ref 1 00000100:00100000:0.0:1789760799.005795:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503957888:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760799.005861:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82145280, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272d0 00010000:00010000:0.0:1789760799.005872:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.005904:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760799.005910:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760799.005918:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.005935:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.017450:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987362 is committed 00002000:00010000:0.0:1789760799.017764:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82145280 -> 82149376 00002000:00010000:0.0:1789760799.017771:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160464 -> 160472 00010000:00010000:0.0:1789760799.017788:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.017803:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.017815:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.017823:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760799.017830:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.017840:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.017867:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.017876:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2c6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82145280->82149375] (req 82145280->82149375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760799.018059:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7e10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2392:x1876699503957888:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12261us (16886us total) trans 4294987362 rc 0/0 00000100:00100000:0.0:1789760799.018072:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20066 00000100:00100000:0.0:1789760799.018084:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729095285233 time 729130382390 state 0x2 ref 0 00000100:00100000:2.0:1789760799.023151:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760799.023627:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958016 00000100:00100000:0.0:1789760799.023643:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958016 00000100:00100000:0.0:1789760799.023716:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729095285233 time 729130382390 state 0x0 ref 1 00000100:00100000:0.0:1789760799.023735:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729130642806 time 729130382390 state 0x1 ref 1 00000100:00100000:0.0:1789760799.023808:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729130642806 time 729148414535 state 0x0 ref 1 00000100:00100000:0.0:1789760799.023822:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5ccdfdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958016:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760799.023890:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82149376, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272d8 00010000:00010000:0.0:1789760799.023902:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.023922:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760799.023926:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760799.023934:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.023949:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.032563:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987363 is committed 00002000:00010000:0.0:1789760799.032810:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82149376 -> 82153472 00002000:00010000:0.0:1789760799.032817:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160472 -> 160480 00010000:00010000:0.0:1789760799.032834:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.032849:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.032860:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.032875:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760799.032891:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.032904:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.032928:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.032937:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82149376->82153471] (req 82149376->82153471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760799.033074:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5ccdfdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958016:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9248us (9931us total) trans 4294987363 rc 0/0 00000100:00100000:0.0:1789760799.033087:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20067 00000100:00100000:0.0:1789760799.033100:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729130642806 time 729148414535 state 0x2 ref 0 00000100:00100000:1.0:1789760799.037948:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760799.038042:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958144 00000100:00100000:3.0:1789760799.038058:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958144 00000100:00100000:3.0:1789760799.038133:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729130642806 time 729148414535 state 0x0 ref 1 00000100:00100000:3.0:1789760799.038153:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729148674951 time 729148414535 state 0x1 ref 1 00000100:00100000:3.0:1789760799.038240:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729148674951 time 729162845398 state 0x0 ref 1 00000100:00100000:3.0:1789760799.038254:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958144:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:3.0:1789760799.038320:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82153472, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272e0 00010000:00010000:3.0:1789760799.038340:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.038363:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760799.038368:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.038376:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.038422:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.046125:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987364 is committed 00002000:00010000:3.0:1789760799.048892:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82153472 -> 82157568 00002000:00010000:3.0:1789760799.048906:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160480 -> 160488 00010000:00010000:3.0:1789760799.048926:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.048948:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.048959:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.048967:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5e020c3e00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:3.0:1789760799.048973:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.048984:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.049008:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.049016:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073747e17/0xfa2da65b083f2c78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82153472->82157567] (req 82153472->82157567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.049266:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958144:12345-192.168.203.58@tcp:4:dd.0 Request processed in 11005us (11328us total) trans 4294987364 rc 0/0 00000100:00100000:3.0:1789760799.049279:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20068 00000100:00100000:3.0:1789760799.049291:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729148674951 time 729162845398 state 0x2 ref 0 00000100:00100000:2.0:1789760799.058100:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760799.059045:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958272 00000100:00100000:1.0:1789760799.059063:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958272 00000100:00100000:1.0:1789760799.059144:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729148674951 time 729162845398 state 0x0 ref 1 00000100:00100000:1.0:1789760799.059234:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729163105814 time 729162845398 state 0x1 ref 1 00000100:00100000:1.0:1789760799.059341:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729163105814 time 729183938145 state 0x0 ref 1 00000100:00100000:1.0:1789760799.059358:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958272:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760799.059464:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82157568, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272e8 00010000:00010000:1.0:1789760799.059579:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.059600:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760799.059605:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.059612:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.059630:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.073236:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987365 is committed 00002000:00010000:1.0:1789760799.077682:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82157568 -> 82161664 00002000:00010000:1.0:1789760799.077696:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160488 -> 160496 00010000:00010000:1.0:1789760799.077760:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.077776:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.077793:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.077802:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f0c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760799.077808:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.077818:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.077846:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.077855:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c988615/0xfa2da65b083f2c7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82157568->82161663] (req 82157568->82161663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.078064:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc696bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958272:12345-192.168.203.58@tcp:4:dd.0 Request processed in 18703us (19971us total) trans 4294987365 rc 0/0 00000100:00100000:1.0:1789760799.078076:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20069 00000100:00100000:1.0:1789760799.078088:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729163105814 time 729183938145 state 0x2 ref 0 00000100:00100000:2.0:1789760799.082611:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760799.082909:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958400 00000100:00100000:0.0:1789760799.082923:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958400 00000100:00100000:0.0:1789760799.082999:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729163105814 time 729183938145 state 0x0 ref 1 00000100:00100000:0.0:1789760799.083021:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729184198561 time 729183938145 state 0x1 ref 1 00000100:00100000:0.0:1789760799.083093:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729184198561 time 729207698167 state 0x0 ref 1 00000100:00100000:0.0:1789760799.083107:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958400:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760799.083186:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82161664, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272f0 00010000:00010000:0.0:1789760799.083198:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.083216:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760799.083221:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760799.083229:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.083246:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.091326:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987366 is committed 00002000:00010000:0.0:1789760799.091819:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82161664 -> 82165760 00002000:00010000:0.0:1789760799.091826:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160496 -> 160504 00010000:00010000:0.0:1789760799.091842:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.091858:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.091868:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.091881:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760799.091887:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.091896:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.091917:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.091926:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82161664->82165759] (req 82161664->82165759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760799.092249:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958400:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9137us (9647us total) trans 4294987366 rc 0/0 00000100:00100000:0.0:1789760799.092270:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20070 00000100:00100000:0.0:1789760799.092282:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729184198561 time 729207698167 state 0x2 ref 0 00000100:00100000:2.0:1789760799.097798:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760799.098272:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958528 00000100:00100000:0.0:1789760799.098286:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958528 00000100:00100000:0.0:1789760799.098365:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729184198561 time 729207698167 state 0x0 ref 1 00000100:00100000:0.0:1789760799.098384:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729207958583 time 729207698167 state 0x1 ref 1 00000100:00100000:0.0:1789760799.098467:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729207958583 time 729223073163 state 0x0 ref 1 00000100:00100000:0.0:1789760799.099083:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958528:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760799.099156:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82165760, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x272f8 00010000:00010000:0.0:1789760799.099179:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.099201:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760799.099206:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760799.099214:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.099232:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.113769:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987367 is committed 00002000:00010000:1.0:1789760799.113938:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82165760 -> 82169856 00002000:00010000:1.0:1789760799.113944:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160504 -> 160512 00010000:00010000:1.0:1789760799.113962:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.113983:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.113993:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114000:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3dba00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760799.114006:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114017:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114041:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114048:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3dcb4f/0xfa2da65b083f2c8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82165760->82169855] (req 82165760->82169855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.114859:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958528:12345-192.168.203.58@tcp:4:dd.0 Request processed in 15095us (16382us total) trans 4294987367 rc 0/0 00000100:00100000:1.0:1789760799.114874:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20071 00000100:00100000:1.0:1789760799.114887:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729207958583 time 729223073163 state 0x2 ref 0 00000100:00100000:2.0:1789760799.119060:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:3.0:1789760799.119211:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958656 00000100:00100000:3.0:1789760799.119224:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958656 00000100:00100000:3.0:1789760799.119303:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729207958583 time 729223073163 state 0x0 ref 1 00000100:00100000:3.0:1789760799.119322:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729223333579 time 729223073163 state 0x1 ref 1 00000100:00100000:3.0:1789760799.119726:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729223333579 time 729244330328 state 0x0 ref 1 00000100:00100000:2.0:1789760799.119818:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958656:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:2.0:1789760799.119885:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82169856, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27300 00010000:00010000:2.0:1789760799.119897:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.119917:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760799.119922:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760799.119929:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.119946:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.128842:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987368 is committed 00002000:00010000:2.0:1789760799.128982:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82169856 -> 82173952 00002000:00010000:2.0:1789760799.128987:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160512 -> 160520 00010000:00010000:2.0:1789760799.129005:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.129017:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.129026:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.129032:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff4f2600 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760799.129036:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.129044:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.129065:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.129070:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b579b7d/0xfa2da65b083f2c94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82169856->82173951] (req 82169856->82173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760799.129183:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958656:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9435us (10130us total) trans 4294987368 rc 0/0 00000100:00100000:2.0:1789760799.129195:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20072 00000100:00100000:2.0:1789760799.129207:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729223333579 time 729244330328 state 0x2 ref 0 00000100:00100000:2.0:1789760799.133575:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760799.133757:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958784 00000100:00100000:1.0:1789760799.133771:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958784 00000100:00100000:1.0:1789760799.133850:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729223333579 time 729244330328 state 0x0 ref 1 00000100:00100000:1.0:1789760799.133870:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729244590744 time 729244330328 state 0x1 ref 1 00000100:00100000:1.0:1789760799.133908:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729244590744 time 729258514105 state 0x0 ref 1 00000100:00100000:1.0:1789760799.133923:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5dffc13050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958784:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760799.133992:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82173952, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27308 00010000:00010000:1.0:1789760799.134003:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.134022:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760799.134027:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.134034:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.134051:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.142660:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987369 is committed 00002000:00010000:1.0:1789760799.142864:0:8436:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82173952 -> 82178048 00002000:00010000:1.0:1789760799.142871:0:8436:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160520 -> 160528 00010000:00010000:1.0:1789760799.142889:0:8436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.142906:0:8436:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.142923:0:8436:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.142932:0:8436:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760799.142938:0:8436:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.142948:0:8436:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.142973:0:8436:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.142982:0:8436:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2c9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82173952->82178047] (req 82173952->82178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.143157:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5dffc13050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503958784:12345-192.168.203.58@tcp:4:dd.0 Request processed in 9229us (9588us total) trans 4294987369 rc 0/0 00000100:00100000:1.0:1789760799.143178:0:8436:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20073 00000100:00100000:1.0:1789760799.143191:0:8436:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729244590744 time 729258514105 state 0x2 ref 0 00000100:00100000:1.0:1789760799.148064:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:0.0:1789760799.148743:0:8434:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503958912 00000100:00100000:0.0:1789760799.148758:0:8434:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503958912 00000100:00100000:0.0:1789760799.148838:0:8434:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729244590744 time 729258514105 state 0x0 ref 1 00000100:00100000:0.0:1789760799.148858:0:8434:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729258774521 time 729258514105 state 0x1 ref 1 00000100:00100000:0.0:1789760799.148901:0:8434:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729258774521 time 729273507631 state 0x0 ref 1 00000100:00100000:0.0:1789760799.148917:0:8434:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5cc7cf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958912:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:0.0:1789760799.148983:0:8434:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82178048, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27310 00010000:00010000:0.0:1789760799.148996:0:8434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.149019:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760799.149024:0:8434:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760799.149031:0:8434:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.149048:0:8434:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.156854:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987370 is committed 00002000:00010000:0.0:1789760799.156997:0:8434:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82178048 -> 82182144 00002000:00010000:0.0:1789760799.157003:0:8434:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160528 -> 160536 00010000:00010000:0.0:1789760799.157021:0:8434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.157036:0:8434:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.157046:0:8434:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.157054:0:8434:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5dff3da400 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:0.0:1789760799.157059:0:8434:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.157069:0:8434:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.157099:0:8434:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.157108:0:8434:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1b8026b/0xfa2da65b083f2ca2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82178048->82182143] (req 82178048->82182143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760799.157292:0:8434:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5cc7cf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503958912:12345-192.168.203.58@tcp:4:dd.0 Request processed in 8371us (9236us total) trans 4294987370 rc 0/0 00000100:00100000:0.0:1789760799.157306:0:8434:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20074 00000100:00100000:0.0:1789760799.157318:0:8434:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729258774521 time 729273507631 state 0x2 ref 0 00000100:00100000:2.0:1789760799.162450:0:3630:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760799.162568:0:15091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503959040 00000100:00100000:1.0:1789760799.162581:0:15091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503959040 00000100:00100000:1.0:1789760799.162653:0:15091:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729258774521 time 729273507631 state 0x0 ref 1 00000100:00100000:1.0:1789760799.162675:0:15091:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729273768047 time 729273507631 state 0x1 ref 1 00000100:00100000:1.0:1789760799.162709:0:15091:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729273768047 time 729287315410 state 0x0 ref 1 00000100:00100000:1.0:1789760799.162723:0:15091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503959040:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760799.162784:0:15091:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82182144, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27318 00010000:00010000:1.0:1789760799.162796:0:15091:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.162817:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760799.162822:0:15091:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.162831:0:15091:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.162847:0:15091:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.173869:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987371 is committed 00002000:00010000:1.0:1789760799.174277:0:15091:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82182144 -> 82186240 00002000:00010000:1.0:1789760799.174283:0:15091:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160536 -> 160544 00010000:00010000:1.0:1789760799.174300:0:15091:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174315:0:15091:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174325:0:15091:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174333:0:15091:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:1.0:1789760799.174339:0:15091:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174349:0:15091:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174374:0:15091:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174382:0:15091:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2ca9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82182144->82186239] (req 82182144->82186239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15091 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.174777:0:15091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503959040:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12048us (12391us total) trans 4294987371 rc 0/0 00000100:00100000:1.0:1789760799.174789:0:15091:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20075 00000100:00100000:1.0:1789760799.174803:0:15091:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729273768047 time 729287315410 state 0x2 ref 0 00000100:00100000:2.0:1789760799.180802:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760799.180898:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503959168 00000100:00100000:1.0:1789760799.180911:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503959168 00000100:00100000:1.0:1789760799.180986:0:8435:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729273768047 time 729287315410 state 0x0 ref 1 00000100:00100000:1.0:1789760799.181007:0:8435:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729287575826 time 729287315410 state 0x1 ref 1 00000100:00100000:1.0:1789760799.181044:0:8435:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729287575826 time 729305649491 state 0x0 ref 1 00000100:00100000:1.0:1789760799.181062:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503959168:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760799.181125:0:8435:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82186240, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27320 00010000:00010000:1.0:1789760799.181137:0:8435:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.181157:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760799.181226:0:8435:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.181236:0:8435:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.181254:0:8435:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.192385:0:8423:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294987372 is committed 00002000:00010000:2.0:1789760799.193068:0:8435:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 82186240 -> 82190336 00002000:00010000:2.0:1789760799.193076:0:8435:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 160544 -> 160552 00010000:00010000:2.0:1789760799.193094:0:8435:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.193109:0:8435:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.193120:0:8435:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.193128:0:8435:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9b5cca1f1c00 already unused, calling callback (ffffffffc0c9b350) 00010000:00010000:2.0:1789760799.193134:0:8435:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.193144:0:8435:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.193181:0:8435:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.193189:0:8435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff9c576f/0xfa2da65b083f2cb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82186240->82190335] (req 82186240->82190335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8435 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760799.193375:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9b5e01be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2391:x1876699503959168:12345-192.168.203.58@tcp:4:dd.0 Request processed in 12307us (12579us total) trans 4294987372 rc 0/0 00000100:00100000:2.0:1789760799.193451:0:8435:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.58@tcp, seq: 20076 00000100:00100000:2.0:1789760799.193466:0:8435:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729287575826 time 729305649491 state 0x2 ref 0 00000100:00100000:2.0:1789760799.202073:0:3631:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.58@tcp (source: 12345-192.168.203.58@tcp) 00000100:00100000:1.0:1789760799.202205:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699503959296 00000100:00100000:1.0:1789760799.202218:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699503959296 00000100:00100000:1.0:1789760799.202296:0:8436:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729287575826 time 729305649491 state 0x0 ref 1 00000100:00100000:1.0:1789760799.202315:0:8436:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729305909907 time 729305649491 state 0x1 ref 1 00000100:00100000:1.0:1789760799.202349:0:8436:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9b5cc6912400 key nid={192.168.203.58@tcp}&opcode={ost_write} rule@0xffff9b5cc7ee5800 gen 0 rate 15 depth 3 token 3 deadline 729305909907 time 729326955296 state 0x0 ref 1 00000100:00100000:1.0:1789760799.202364:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9b5e01be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:79662a0f-af46-4cf0-8a56-ded8e9c54f3e+5:2390:x1876699503959296:12345-192.168.203.58@tcp:4:dd.0 00002000:00010000:1.0:1789760799.202492:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 82190336, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x27328 00010000:00010000:1.0:1789760799.202525:0:8436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2cb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.202551:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760799.202556:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.202564:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82190336->82194431] (req 82190336->82194431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.202581:0:8436:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f202864/0xfa2da65b083f2cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [82190336->82194431] (req 82190336->82194431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 Debug log: 57144 lines, 57144 kept, 0 dropped, 0 bad.